-- Logs begin at Fri 2022-10-28 18:29:24 UTC, end at Fri 2022-10-28 19:19:48 UTC. -- Oct 28 18:52:28.841715 dv-cinder-862365-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 28 18:52:28.851117 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 28 18:52:28 2022] *** Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: nodename: dv-cinder-862365-1-iscsi Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: machine: x86_64 Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: clock source: unix Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: pcre jit disabled Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: detected number of CPU cores: 4 Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: current working directory: / Oct 28 18:52:28.852418 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: detected binary path: /usr/bin/uwsgi-core Oct 28 18:52:28.853060 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: your processes number limit is 47680 Oct 28 18:52:28.853060 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: your memory page size is 4096 bytes Oct 28 18:52:28.853060 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: detected max file descriptor number: 2048 Oct 28 18:52:28.853060 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: lock engine: pthread robust mutexes Oct 28 18:52:28.853060 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: thunder lock: enabled Oct 28 18:52:28.853060 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 28 18:52:28.853060 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 28 18:52:28.893305 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: Python main interpreter initialized at 0x56464f14a9c0 Oct 28 18:52:28.893305 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: python threads support enabled Oct 28 18:52:28.893305 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: your server socket listen backlog is limited to 100 connections Oct 28 18:52:28.893305 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: your mercy for graceful operations on workers is 90 seconds Oct 28 18:52:28.893668 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: mapped 403077 bytes (393 KB) for 2 cores Oct 28 18:52:28.893740 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: *** Operational MODE: preforking *** Oct 28 18:52:28.894000 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: *** uWSGI is running in multiple interpreter mode *** Oct 28 18:52:28.894000 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: spawned uWSGI master process (pid: 81351) Oct 28 18:52:28.894203 dv-cinder-862365-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 28 18:52:28.895210 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: spawned uWSGI worker 1 (pid: 81363, cores: 1) Oct 28 18:52:28.895210 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: spawned uWSGI worker 2 (pid: 81364, cores: 1) Oct 28 18:52:28.895797 dv-cinder-862365-1-iscsi devstack@keystone.service[81351]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 28 18:52:33.168759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 18:52:33.168759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: from cryptography.utils import int_from_bytes Oct 28 18:52:33.169549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 18:52:33.169549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: from cryptography.utils import int_from_bytes Oct 28 18:52:33.169668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 18:52:33.169668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: from cryptography.utils import int_from_bytes Oct 28 18:52:33.169668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 18:52:33.169668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: from cryptography.utils import int_from_bytes Oct 28 18:52:33.219335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 18:52:33.219504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 18:52:33.219701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] command line args: [] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 18:52:33.219969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 18:52:33.220148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ================================================================================ {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 18:52:33.220401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] admin_token = **** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.220571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 18:52:33.220661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] config_dir = [] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.220769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 18:52:33.220849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.220962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] command line args: [] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 18:52:33.221044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] config_source = [] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.221157 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] debug = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.221240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 18:52:33.221357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.221683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ================================================================================ {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 18:52:33.221683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] admin_token = **** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.221881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.221983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] config_dir = [] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] config_source = [] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] debug = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] list_limit = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] log_config_append = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.222999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] log_dir = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] log_file = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] list_limit = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] log_config_append = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.223997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.224206 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.224287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] log_dir = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.224408 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.224485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] log_file = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.224604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.224688 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.224802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.224876 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.224994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.225963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.226088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.226214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.226297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.226457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.226545 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.226706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.226826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.226940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.227194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.227309 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.227438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] publish_errors = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.227560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.227671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.227789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.227911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.228014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.228178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.228302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.228423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] publish_errors = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.228541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.228652 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.228769 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] standard_threads = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.228899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.229022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.229135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.229247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.229358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.229481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.229602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.229715 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] use_journal = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.229715 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] use_json = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.229932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] standard_threads = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.230043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] use_stderr = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.230155 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.230320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] use_syslog = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.230442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.230560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.230673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.230791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.230910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] use_journal = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.230910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] use_json = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.231147 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.231266 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] use_stderr = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.231380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.231497 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] use_syslog = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.231619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.231741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 18:52:33.231853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.231964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.232076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.232194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.232306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.233048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.233163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.233274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.233387 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.233503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.233615 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.233735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.233847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.233964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.234076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.234193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.234312 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.234432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.234551 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.234669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.234789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.234902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.235032 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.235167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.235288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.235407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.235520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.235632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.235742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.235862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.235975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.236086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.236197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.236306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.236492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.236608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.236719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.236829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.236939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.237049 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.237171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.237283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.237394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.237505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.237616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.237727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.237845 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.237956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.238067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.238177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.238289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.238399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.238517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.238629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.238746 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.238857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.238967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.239104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.239227 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.239346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.239457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.239567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.239678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.239794 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.239920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.240032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.240148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.240260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.240370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.240486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.240627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.240742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.240859 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.240976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.241088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.241198 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.241319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.241431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.241541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.241651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.241767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.241885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.242066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.242183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.242302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.242414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.242530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.242647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.242766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.242885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.243050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.243166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.243283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.243394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.243514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.243625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.243735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.243845 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.243956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.244075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.244172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.244272 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.244366 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.244466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.244599 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.244709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.244806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.244906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.245003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.245134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.245258 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.245370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.245483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.245595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.245705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.245824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.245937 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.246051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.246175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.246289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.246406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.246520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.246631 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.246752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.246866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.247005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.247134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.247258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.247372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.247485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.247597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.247717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.247830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.247941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.248660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.248776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.248890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.249002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.249116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.249236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.249340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.249340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.249557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.249680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.249793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.249905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.250017 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.250129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.250243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.250364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.250478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.250591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.250703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.250816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.250928 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.251099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.251215 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.251329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.251447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.251561 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.251679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.251800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.251913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.252026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.252145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.252261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.252376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.252498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.252617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.252737 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.252850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.252963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.253074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.253194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.253315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.253429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.253547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.253660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.253772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.253960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.254075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.254188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.254299 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.254412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.254533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.254658 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.254780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.254894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.255020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.255134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.255134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.255357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.255481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.255481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.255708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.255821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.255821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.256102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.256229 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.256344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.256458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.256458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] auth.external = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.256677 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.256791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.256791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.257011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.257124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.257124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] auth.password = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.257334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.257478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] auth.token = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.257478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.257704 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.257826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.257826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.258040 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.258153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.258153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.258366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.258487 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.258609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.258722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.258836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] credential.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.259081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.259202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.259329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.259562 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.259679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.259817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.260027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] auth.external = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.260145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.260270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.260461 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.260579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.260750 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.260986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.261103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.261226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.261455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.261573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.261737 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.261888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] auth.password = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.262006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.262228 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.262351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] auth.token = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.262474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.262707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] federation.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.262823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.262937 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.263200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.263318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.263431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.263686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.263803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.263917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.264038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.264152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.264265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.264399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.264634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.264758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.264875 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.264989 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.265102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.265215 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] credential.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.265328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.265449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.265564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.265677 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.265791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.265905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.266018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.266140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.266254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.266368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.266481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.266594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.266707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.266828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.266944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.267108 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.267226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.267340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.267453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.267828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.267947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.268119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.268237 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.268358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.268471 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.268593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.268707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.268821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.268934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.269055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.269175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.269361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.269478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] federation.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.269591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.269695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.269798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.269900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.270021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.270138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.270259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.270373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.270486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.270599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.270720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.270834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.270947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.271121 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.271238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.271358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.271481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.271596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.271709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.271822 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.271941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.272087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.272212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.272327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.272440 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.272553 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.272666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.272779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.272900 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.273014 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.273127 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.273241 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.273358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.273472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.273594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.273709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.273822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.273935 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.274049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.274161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.274283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.274398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.274511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.274624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.274737 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.274850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.274850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275195 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.275995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.276213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.276327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.276440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.276552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.276663 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.276776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.276896 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.277010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.277121 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.277234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.277345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.277459 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.277578 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.277692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.277804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.277916 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.278036 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.278036 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.278254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.278254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.278466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.278466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.278676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.278788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.278909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.279068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.279183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.279295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.279407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.279407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.279629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.279744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.279856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.279969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.280988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.281198 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.281311 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.281423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.281536 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.281657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.281771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.281883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.281995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.282107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.282219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.282344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.282458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.282571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.282685 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.282798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.282910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.283047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.283163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.283279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.283392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.283505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.283617 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.283739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.283853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.283965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.284078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.284197 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.284311 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.284431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.284545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.284657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.284776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.284889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.285002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.285122 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.285236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.285348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.285461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.285573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.285686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.285806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.285920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.286032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.286145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] resource.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.286257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.286369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.286489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.286603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.286715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.286827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.286941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.287094 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.287219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.287334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.287446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.287558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.287669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.287781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.287901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.288015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.288127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.288239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.288350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.288471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] role.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.288584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.288696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] role.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.288809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.288921 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.289033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.289153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.289267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.289386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.289500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.289613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.289724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.289844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.289958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.290072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.290185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.290297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.290409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.290530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.290643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.290755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.290867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.290992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.291107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.291297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.291413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.291526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.291644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.291758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.291870 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.291991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.292105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.292105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.292316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.292428 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] resource.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.292540 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.292660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.292773 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.292886 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.292998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.293111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.293228 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.293351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.293466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.293579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.293692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.293805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.293918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.294039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.294154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.294268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.294381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.294494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.294606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.294728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] role.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.295369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.295489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] role.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.295604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.295717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.296081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.296212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.296328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.296442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.296555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.296668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.296781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.296903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.297017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.297130 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.297243 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.297356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.297477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] token.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.297592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.297705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.297818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.297931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.298043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.298165 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.298279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.298392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.298507 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.298621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.298733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.298854 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.298969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.299103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.299103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.299315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.299428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.299554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.299669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.299783 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.299896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.300008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.300121 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.300242 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.300356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.300470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.300585 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.300698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.300811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.300927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.301037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.301151 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.301265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.301379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.301492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.connection = **** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.301606 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.301727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.301842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.301955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.302068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.302181 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.302294 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.302416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.302530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.302644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.302757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.302870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.303028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.303159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.303275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.303389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.303503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.303616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.303737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] token.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.303860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.303974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.304088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.304201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.304314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.304427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.304548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.304662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.304776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.304889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 18:52:33.305910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.305910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.306221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.306221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.306511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.306742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.307005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.307275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.307534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.connection = **** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.307778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.308024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.308262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.308501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.308735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.308970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.309203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.309437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.309679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.309915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.310158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.310392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.311226 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.311467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.311711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.311941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.312178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.312412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 18:52:33.312581 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 18:52:33.334771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.335254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.337153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.337390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.337618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.343630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.345884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.373757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.373967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.374055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.374202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.425513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.425904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.473567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.473798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.474322 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.474560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.629173 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.629358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.629462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.629541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.629614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.629709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.629816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.629897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.630034 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.630125 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.630249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.630332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.653521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.653628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.653761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.653889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.653969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.654042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.654193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.654798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.654939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.655051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.655181 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.655263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.655396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.655477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.666445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.666690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.679909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.680020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.680087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.680236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.680346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.680456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.680574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.680682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.680793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.680884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.681011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.681093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.681226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.681306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 18:52:33.698020 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.698123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.698229 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 28 18:52:33.698336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 28 18:52:33.698571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.698665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.698807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 28 18:52:33.698899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 28 18:52:33.699070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.699154 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.699336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.699430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.700012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.700103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.700178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.700250 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.709360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.709465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.709641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.709723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.709914 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.709995 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.710199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.710280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.717650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.717768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.720592 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.720696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.720879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.720960 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.721165 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.721245 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.721447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.721529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.721727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.721809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.721992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.722073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.722259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.722340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.722530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.722610 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.722915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.723022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.723221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.723302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.723490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.723569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.746309 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.746469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.747779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.747898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.748093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.748204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.754546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.754656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.754836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.754921 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.755144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.755232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.761562 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.761797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.761882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.762089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.766017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.766271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.766356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.766566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.766649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.766853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.766936 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.767161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.767245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.767436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.767531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.767960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.768044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.768235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.768318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.768512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.784152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.784431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.784518 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.784727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.788268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.788571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.791469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.791771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.791855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.792072 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.796481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.796807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.799615 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.800053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.802808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.803133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.803290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.803419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.803590 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.803702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.803879 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.803997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.804169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.804284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.804467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.804757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.819890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.820178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.820403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.820487 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.820693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.820789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.820962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.821233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.827526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.828111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.829063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.829646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.830614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.830890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.831225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.831504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.833535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.833819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.834211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.834294 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.834498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.834590 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.834789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.834899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.835212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.835506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.846308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.846602 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.846999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.847109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.847286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.847397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.847581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.847774 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.847858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.848057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.848149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.848352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.848638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.848918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.863852 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.864808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.866714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.867722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.869616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.870589 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.871037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.872140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.872652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.872946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.873240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.873762 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.874049 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.874337 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.879543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.880666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.881005 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.882128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.883997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.884290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.884575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.884852 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.885114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.885405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.885687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.885964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.893354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.893645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.893927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.894519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.894812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.895122 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.901614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.901908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.902180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.902456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.902733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.902814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 18:52:33.903023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.903123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.903302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.903385 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.903590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.903673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.903865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.903956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.904134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.904278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.904960 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.905045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.905231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.905314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.905499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.905627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.905763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.905847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.906046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.906319 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.906591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.906853 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 18:52:33.938475 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56464f14a9c0 pid: 81364 (default app) Oct 28 18:52:33.939486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56464f14a9c0 pid: 81363 (default app) Oct 28 18:52:33.941705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5833b80-1fe8-485a-8d3a-d3cfc8050860 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:33.942000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5833b80-1fe8-485a-8d3a-d3cfc8050860 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:33.942272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5833b80-1fe8-485a-8d3a-d3cfc8050860 None None] PATH_INFO: `/v3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:33.946420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Fri Oct 28 18:52:33 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:52:40.637985 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe363e9-c34c-4a25-aec5-0b9ebe439c37 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:40.638538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe363e9-c34c-4a25-aec5-0b9ebe439c37 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:40.638648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe363e9-c34c-4a25-aec5-0b9ebe439c37 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:40.643673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:52:40.648742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:40.649300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:40.649300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:40.718491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: INFO dbcounter [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] Registered counter for database keystone Oct 28 18:52:40.737808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG oslo_db.sqlalchemy.engines [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81364) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 28 18:52:40.743962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writer thread running {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 18:52:40.920409 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG passlib.handlers.bcrypt [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81364) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 28 18:52:40.920702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG passlib.handlers.bcrypt [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81364) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 28 18:52:40.978110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:40.979738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:41.103560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 28 18:52:41.560077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe363e9-c34c-4a25-aec5-0b9ebe439c37 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:41.629977 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: INFO dbcounter [None req-5fe363e9-c34c-4a25-aec5-0b9ebe439c37 None None] Registered counter for database keystone Oct 28 18:52:41.651280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG oslo_db.sqlalchemy.engines [None req-5fe363e9-c34c-4a25-aec5-0b9ebe439c37 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81363) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 28 18:52:41.657352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writer thread running {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 18:52:41.777054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe363e9-c34c-4a25-aec5-0b9ebe439c37 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:41.781958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a2f24-e1b7-4834-b027-aaff95b2df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:41.782583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a2f24-e1b7-4834-b027-aaff95b2df74 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:41.782866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a2f24-e1b7-4834-b027-aaff95b2df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:41.783166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a2f24-e1b7-4834-b027-aaff95b2df74 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:41.791325 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a2f24-e1b7-4834-b027-aaff95b2df74 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:41.842479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING keystone.common.rbac_enforcer.enforcer [None req-576a2f24-e1b7-4834-b027-aaff95b2df74 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 28 18:52:41.842773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a2f24-e1b7-4834-b027-aaff95b2df74 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:41.852229 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Fri Oct 28 18:52:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 28 18:52:41.858164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:41.875042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81f21c-a1c8-44a0-a256-4e11db274ed7 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:41.879413 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae07ed2-0dfb-4685-b5b8-df142267c809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:41.880513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae07ed2-0dfb-4685-b5b8-df142267c809 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:41.880790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae07ed2-0dfb-4685-b5b8-df142267c809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:41.881056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae07ed2-0dfb-4685-b5b8-df142267c809 admin admin] PATH_INFO: `/v3/projects/b28e4680a856417f99f9b94c78ac743d` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:41.884780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae07ed2-0dfb-4685-b5b8-df142267c809 admin admin] RBAC: Authorizing `identity:get_project(project_id=b28e4680a856417f99f9b94c78ac743d)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:41.934488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fae07ed2-0dfb-4685-b5b8-df142267c809 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 28 18:52:41.934829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae07ed2-0dfb-4685-b5b8-df142267c809 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:41.939849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Fri Oct 28 18:52:41 2022] GET /identity/v3/projects/b28e4680a856417f99f9b94c78ac743d => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:52:44.021645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f974e-6164-4e0f-89a4-34dc04b8afbb None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:44.022149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f974e-6164-4e0f-89a4-34dc04b8afbb None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:44.022231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f974e-6164-4e0f-89a4-34dc04b8afbb None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:44.026277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:52:44.032117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b12004-327b-425f-9ccc-437b4a25b62a None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:44.032507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b12004-327b-425f-9ccc-437b4a25b62a None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:44.032655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b12004-327b-425f-9ccc-437b4a25b62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:44.052186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d2b12004-327b-425f-9ccc-437b4a25b62a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:44.053673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d2b12004-327b-425f-9ccc-437b4a25b62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:44.061924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 28 18:52:44.324439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f974e-6164-4e0f-89a4-34dc04b8afbb None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:44.336971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f974e-6164-4e0f-89a4-34dc04b8afbb None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:44.341163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7929a8-fb38-459c-a310-9451880d63f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:44.341781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7929a8-fb38-459c-a310-9451880d63f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:44.342064 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7929a8-fb38-459c-a310-9451880d63f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:44.342338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7929a8-fb38-459c-a310-9451880d63f5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:44.349872 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7929a8-fb38-459c-a310-9451880d63f5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:44.351678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7929a8-fb38-459c-a310-9451880d63f5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:44.360289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Fri Oct 28 18:52:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 18:52:44.365259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b12004-327b-425f-9ccc-437b4a25b62a None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:44.374652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b12004-327b-425f-9ccc-437b4a25b62a None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:44.378884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c1eba-f114-4df6-8c61-d82bfd0e4a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:44.379524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c1eba-f114-4df6-8c61-d82bfd0e4a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:44.379800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c1eba-f114-4df6-8c61-d82bfd0e4a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:44.380064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c1eba-f114-4df6-8c61-d82bfd0e4a9b admin admin] PATH_INFO: `/v3/users/d171ff8c81304d799c869856dff7ee6d` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:44.381337 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c1eba-f114-4df6-8c61-d82bfd0e4a9b admin admin] RBAC: Authorizing `identity:get_user(user_id=d171ff8c81304d799c869856dff7ee6d)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:44.383179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c1eba-f114-4df6-8c61-d82bfd0e4a9b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:44.384664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Fri Oct 28 18:52:44 2022] GET /identity/v3/users/d171ff8c81304d799c869856dff7ee6d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:52:46.578966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f46bcb-4937-4417-98b9-16e935711db4 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:46.579559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f46bcb-4937-4417-98b9-16e935711db4 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:46.579559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f46bcb-4937-4417-98b9-16e935711db4 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:46.581043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:52:46.585905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ec9b0-4519-4119-ac68-48eb1886ffbc None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:46.586301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ec9b0-4519-4119-ac68-48eb1886ffbc None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:46.586454 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ec9b0-4519-4119-ac68-48eb1886ffbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:46.605519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39245be9-ffe6-4bd9-8e85-b79db3ae095f None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:46.605791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-bb1ec9b0-4519-4119-ac68-48eb1886ffbc None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:46.605886 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39245be9-ffe6-4bd9-8e85-b79db3ae095f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:46.606054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39245be9-ffe6-4bd9-8e85-b79db3ae095f None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:46.606795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-bb1ec9b0-4519-4119-ac68-48eb1886ffbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:46.608024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 6/11] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:52:46.613194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100fcc9-6fce-4006-ab20-869925ed4b72 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:46.613470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100fcc9-6fce-4006-ab20-869925ed4b72 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:46.613733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100fcc9-6fce-4006-ab20-869925ed4b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:46.656308 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG passlib.handlers.bcrypt [None req-b100fcc9-6fce-4006-ab20-869925ed4b72 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81363) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 28 18:52:46.656692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG passlib.handlers.bcrypt [None req-b100fcc9-6fce-4006-ab20-869925ed4b72 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81363) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 28 18:52:46.690595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:52:46.705445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-b100fcc9-6fce-4006-ab20-869925ed4b72 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:46.707065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b100fcc9-6fce-4006-ab20-869925ed4b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:46.808663 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:52:46.961688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ec9b0-4519-4119-ac68-48eb1886ffbc None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:46.985872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:47.067330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ec9b0-4519-4119-ac68-48eb1886ffbc None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:47.073557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100fcc9-6fce-4006-ab20-869925ed4b72 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:47.101408 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347308bb-eb1a-425a-86dc-bc39318e39e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:47.102037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347308bb-eb1a-425a-86dc-bc39318e39e3 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:47.102313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347308bb-eb1a-425a-86dc-bc39318e39e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:47.102590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347308bb-eb1a-425a-86dc-bc39318e39e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:47.113837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347308bb-eb1a-425a-86dc-bc39318e39e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-347308bb-eb1a-425a-86dc-bc39318e39e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:52:47.122611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:52:47.123756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:52:47.124613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 7/14] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:52:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:52:47.129507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347308bb-eb1a-425a-86dc-bc39318e39e3 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:47.176010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100fcc9-6fce-4006-ab20-869925ed4b72 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:47.178247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:47.210899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:47.213505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:47.214556 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:47.214713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:47.215004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:47.215975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:47.217403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:47.228376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.sql.core [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 28 18:52:47.228376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 28 18:52:47.228376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [parameters: {'id': '5415cf27b9bf443e8ed961f067cebe43', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 28 18:52:47.228376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 28 18:52:47.229912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347308bb-eb1a-425a-86dc-bc39318e39e3 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:52:47.247044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application result.read() Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 28 18:52:47.248530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application yield resource Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.session.rollback() Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.force_reraise() Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise self.value Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application session.commit() Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 28 18:52:47.249611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._prepare_impl() Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.session.flush() Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._flush(objects) Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application compat.raise_( Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application flush_context.execute() Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rec.execute(self) Oct 28 18:52:47.250678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 28 18:52:47.251599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application result.read() Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:52:47.252329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [parameters: {'id': '5415cf27b9bf443e8ed961f067cebe43', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 28 18:52:47.253111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:52:47.254757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 8/15] 10.222.0.58 () {60 vars in 1137 bytes} [Fri Oct 28 18:52:47 2022] POST /identity/v3/domains => generated 190 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:52:47.256133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:47.265413 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52615961-518f-4a9c-8885-ccb78d133a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:47.266052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52615961-518f-4a9c-8885-ccb78d133a4b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:47.266333 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52615961-518f-4a9c-8885-ccb78d133a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:47.266607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52615961-518f-4a9c-8885-ccb78d133a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:47.267319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52615961-518f-4a9c-8885-ccb78d133a4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:52:47.267625 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52615961-518f-4a9c-8885-ccb78d133a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:47.269276 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52615961-518f-4a9c-8885-ccb78d133a4b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:47.281162 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 8/16] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:52:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:52:47.286594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52615961-518f-4a9c-8885-ccb78d133a4b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:47.353155 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b94be-82ae-45f5-8e31-7dbf68fcd4fc None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:47.383109 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52615961-518f-4a9c-8885-ccb78d133a4b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:47.387173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48a8f-915b-4a2a-bb8f-8dd641e6de19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:47.388026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48a8f-915b-4a2a-bb8f-8dd641e6de19 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:47.388355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48a8f-915b-4a2a-bb8f-8dd641e6de19 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:47.389129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48a8f-915b-4a2a-bb8f-8dd641e6de19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:47.407290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad48a8f-915b-4a2a-bb8f-8dd641e6de19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:47.408756 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad48a8f-915b-4a2a-bb8f-8dd641e6de19 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:47.417257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0307f-009e-425f-a46e-27e989677b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:47.417887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0307f-009e-425f-a46e-27e989677b93 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:47.418163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0307f-009e-425f-a46e-27e989677b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:47.418434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0307f-009e-425f-a46e-27e989677b93 None admin] PATH_INFO: `/v3/users/d171ff8c81304d799c869856dff7ee6d` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:47.420156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0307f-009e-425f-a46e-27e989677b93 None admin] RBAC: Authorizing `identity:get_user(user_id=d171ff8c81304d799c869856dff7ee6d)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:47.421626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0307f-009e-425f-a46e-27e989677b93 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:47.423211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 9/17] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:52:47 2022] GET /identity/v3/users/d171ff8c81304d799c869856dff7ee6d => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-9ad48a8f-915b-4a2a-bb8f-8dd641e6de19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:52:47.427508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:52:47.428631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:52:47.429974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:52:47.429974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:52:47.429974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:47.430176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0307f-009e-425f-a46e-27e989677b93 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:47.430254 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 9/18] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:52:47.434101 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48a8f-915b-4a2a-bb8f-8dd641e6de19 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:47.443569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:47.456351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:47.528260 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0307f-009e-425f-a46e-27e989677b93 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:47.534196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48a8f-915b-4a2a-bb8f-8dd641e6de19 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:47.562502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27161f1d-98e0-4e33-8d44-9a3257975bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:47.563152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27161f1d-98e0-4e33-8d44-9a3257975bbf None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:47.563435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27161f1d-98e0-4e33-8d44-9a3257975bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:47.563706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27161f1d-98e0-4e33-8d44-9a3257975bbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:47.564813 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27161f1d-98e0-4e33-8d44-9a3257975bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:47.566274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27161f1d-98e0-4e33-8d44-9a3257975bbf None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:47.567491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a00c47-d12e-4e53-95e4-544ea5312ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:47.567842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 10/19] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:52:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:52:47.568095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a00c47-d12e-4e53-95e4-544ea5312ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:47.568371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a00c47-d12e-4e53-95e4-544ea5312ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:47.568639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a00c47-d12e-4e53-95e4-544ea5312ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:47.569273 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a00c47-d12e-4e53-95e4-544ea5312ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:52:47.569571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a00c47-d12e-4e53-95e4-544ea5312ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:47.570950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a00c47-d12e-4e53-95e4-544ea5312ec4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:47.573444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27161f1d-98e0-4e33-8d44-9a3257975bbf None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:47.588609 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 10/20] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:52:47.657668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:47.678163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27161f1d-98e0-4e33-8d44-9a3257975bbf None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:47.713379 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905dd3e-35fe-484f-9b67-dc343aa94a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:47.714049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905dd3e-35fe-484f-9b67-dc343aa94a5f None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:47.714322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905dd3e-35fe-484f-9b67-dc343aa94a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:47.714590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905dd3e-35fe-484f-9b67-dc343aa94a5f None admin] PATH_INFO: `/v3/domains/default/users/d171ff8c81304d799c869856dff7ee6d/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:47.716600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905dd3e-35fe-484f-9b67-dc343aa94a5f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d171ff8c81304d799c869856dff7ee6d, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:47.718051 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905dd3e-35fe-484f-9b67-dc343aa94a5f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:47.835935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 11/21] 10.222.0.58 () {58 vars in 1361 bytes} [Fri Oct 28 18:52:47 2022] PUT /identity/v3/domains/default/users/d171ff8c81304d799c869856dff7ee6d/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 18:52:50.059633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b3844-ab26-4232-bd6e-10c8290910e6 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:50.060717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b3844-ab26-4232-bd6e-10c8290910e6 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:50.060717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b3844-ab26-4232-bd6e-10c8290910e6 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:50.062067 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 11/22] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:52:50.082284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd272b7-25a6-48b1-b3f8-e7bab7b950ab None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:50.082789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd272b7-25a6-48b1-b3f8-e7bab7b950ab None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:50.082859 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd272b7-25a6-48b1-b3f8-e7bab7b950ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:50.107818 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-8dd272b7-25a6-48b1-b3f8-e7bab7b950ab None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:50.109328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8dd272b7-25a6-48b1-b3f8-e7bab7b950ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:50.198709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 12/23] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:52:50.727920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b3844-ab26-4232-bd6e-10c8290910e6 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:50.742493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825eb99-6031-4d3c-b4c4-837f7685e295 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:50.743152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825eb99-6031-4d3c-b4c4-837f7685e295 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:50.743619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825eb99-6031-4d3c-b4c4-837f7685e295 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:50.750418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:52:50.756275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764777e3-6946-4ad9-9948-15ba2f380105 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:50.757313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764777e3-6946-4ad9-9948-15ba2f380105 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:50.758641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764777e3-6946-4ad9-9948-15ba2f380105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:50.797289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-764777e3-6946-4ad9-9948-15ba2f380105 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:50.800192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-764777e3-6946-4ad9-9948-15ba2f380105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:50.864227 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:50.892644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b3844-ab26-4232-bd6e-10c8290910e6 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:50.914402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:50.920926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 14/25] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:52:50.922746 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436a258-5328-479c-9f5c-63c3540bc824 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:50.923223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436a258-5328-479c-9f5c-63c3540bc824 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:50.923644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436a258-5328-479c-9f5c-63c3540bc824 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:50.925138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 15/26] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:52:50.926329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e41cf-8e36-4689-9120-45afccbd43db None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:50.926947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e41cf-8e36-4689-9120-45afccbd43db None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:50.927463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e41cf-8e36-4689-9120-45afccbd43db None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:50.933026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 16/27] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:52:50.934430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705cfcb-185e-4dc2-b7e9-8bdaf1b98144 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:50.934931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705cfcb-185e-4dc2-b7e9-8bdaf1b98144 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:50.936008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705cfcb-185e-4dc2-b7e9-8bdaf1b98144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:50.952010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816dc4a0-756e-48f5-9860-970afe4dc66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:50.952659 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dc4a0-756e-48f5-9860-970afe4dc66b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:50.952935 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dc4a0-756e-48f5-9860-970afe4dc66b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:50.953204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816dc4a0-756e-48f5-9860-970afe4dc66b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:50.956803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-0705cfcb-185e-4dc2-b7e9-8bdaf1b98144 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:50.958221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0705cfcb-185e-4dc2-b7e9-8bdaf1b98144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:50.970048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816dc4a0-756e-48f5-9860-970afe4dc66b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:50.971839 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816dc4a0-756e-48f5-9860-970afe4dc66b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-816dc4a0-756e-48f5-9860-970afe4dc66b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:52:50.995875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:52:50.996771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:52:50.998113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:52:50.998113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:52:50.998113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:51.000025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 12/28] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:52:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:52:51.001407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365df88-1572-49dc-a0b3-9679f172d2f6 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:51.001906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365df88-1572-49dc-a0b3-9679f172d2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:51.002492 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365df88-1572-49dc-a0b3-9679f172d2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:51.037294 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e365df88-1572-49dc-a0b3-9679f172d2f6 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:51.038487 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e365df88-1572-49dc-a0b3-9679f172d2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:51.083327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 17/29] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:52:51.085070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705cfcb-185e-4dc2-b7e9-8bdaf1b98144 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:51.140787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 13/30] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:52:51.238789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365df88-1572-49dc-a0b3-9679f172d2f6 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:51.292779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705cfcb-185e-4dc2-b7e9-8bdaf1b98144 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:51.350063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:51.361406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30153d36-3ab5-4c1f-b07e-ee9d766375ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:51.362614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30153d36-3ab5-4c1f-b07e-ee9d766375ae None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:51.363225 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30153d36-3ab5-4c1f-b07e-ee9d766375ae None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:51.364666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30153d36-3ab5-4c1f-b07e-ee9d766375ae None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:51.366180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30153d36-3ab5-4c1f-b07e-ee9d766375ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:52:51.367249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30153d36-3ab5-4c1f-b07e-ee9d766375ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:51.367377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:51.369076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30153d36-3ab5-4c1f-b07e-ee9d766375ae None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:51.409569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 18/31] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:52:51.416560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30153d36-3ab5-4c1f-b07e-ee9d766375ae None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:51.467888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365df88-1572-49dc-a0b3-9679f172d2f6 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:51.514846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e092db-2d81-4c28-aa48-8e06bac4a6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:51.515830 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e092db-2d81-4c28-aa48-8e06bac4a6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:51.516252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e092db-2d81-4c28-aa48-8e06bac4a6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:51.516664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e092db-2d81-4c28-aa48-8e06bac4a6b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:51.531519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e092db-2d81-4c28-aa48-8e06bac4a6b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-71e092db-2d81-4c28-aa48-8e06bac4a6b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:52:51.540476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:52:51.541579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:52:51.542530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:52:51.545589 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 14/32] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:52:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:52:51.546832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e092db-2d81-4c28-aa48-8e06bac4a6b4 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:51.593787 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30153d36-3ab5-4c1f-b07e-ee9d766375ae None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:51.636418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d6f4f-46b8-439d-b8f8-4ada68b3944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:51.637330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d6f4f-46b8-439d-b8f8-4ada68b3944e None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:51.637777 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d6f4f-46b8-439d-b8f8-4ada68b3944e None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:51.638810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d6f4f-46b8-439d-b8f8-4ada68b3944e None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:51.639651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d6f4f-46b8-439d-b8f8-4ada68b3944e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:51.641281 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d6f4f-46b8-439d-b8f8-4ada68b3944e None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:51.671105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e092db-2d81-4c28-aa48-8e06bac4a6b4 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:51.682033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 19/33] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:52:51 2022] POST /identity/v3/projects => generated 301 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:52:51.683564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d6f4f-46b8-439d-b8f8-4ada68b3944e None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:51.700238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:51.716644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1d6e3-784d-4b5a-993d-cd3231038821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:51.717744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1d6e3-784d-4b5a-993d-cd3231038821 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:51.718296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1d6e3-784d-4b5a-993d-cd3231038821 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:51.718767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1d6e3-784d-4b5a-993d-cd3231038821 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:51.719937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1d6e3-784d-4b5a-993d-cd3231038821 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:51.721911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1d6e3-784d-4b5a-993d-cd3231038821 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:51.743691 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:51.810859 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d6f4f-46b8-439d-b8f8-4ada68b3944e None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:51.847696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519597e1-85d4-4c55-a706-b0ab0f4dee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:51.848315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519597e1-85d4-4c55-a706-b0ab0f4dee2f None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:51.848599 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519597e1-85d4-4c55-a706-b0ab0f4dee2f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:51.848875 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519597e1-85d4-4c55-a706-b0ab0f4dee2f None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:51.849552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519597e1-85d4-4c55-a706-b0ab0f4dee2f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:51.850944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519597e1-85d4-4c55-a706-b0ab0f4dee2f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:51.857147 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 15/34] 10.222.0.58 () {60 vars in 1131 bytes} [Fri Oct 28 18:52:51 2022] POST /identity/v3/roles => generated 225 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:52:51.858104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1d6e3-784d-4b5a-993d-cd3231038821 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:51.869719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 20/35] 10.222.0.58 () {60 vars in 1131 bytes} [Fri Oct 28 18:52:51 2022] POST /identity/v3/roles => generated 219 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:52:51.872042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64d70d-6455-4fd1-a026-247f38294732 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:51.872042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64d70d-6455-4fd1-a026-247f38294732 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:51.872042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64d70d-6455-4fd1-a026-247f38294732 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:51.878150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 21/36] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:52:51.878150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec3f4b-1307-4f93-bc47-2e6d93a95027 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:51.878150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec3f4b-1307-4f93-bc47-2e6d93a95027 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:51.878150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec3f4b-1307-4f93-bc47-2e6d93a95027 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:51.878150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 22/37] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:52:51.881428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1049357-659f-4ff5-b2d9-8b86c4d59c4b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:51.881428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1049357-659f-4ff5-b2d9-8b86c4d59c4b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:51.881428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1049357-659f-4ff5-b2d9-8b86c4d59c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:51.900669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b1049357-659f-4ff5-b2d9-8b86c4d59c4b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:51.901914 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b1049357-659f-4ff5-b2d9-8b86c4d59c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:52.002871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1d6e3-784d-4b5a-993d-cd3231038821 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:52.007701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 23/38] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:52:52.010660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce69283-fdc4-4fc0-bbc7-0a79e776e0dd None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:52.011044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce69283-fdc4-4fc0-bbc7-0a79e776e0dd None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:52.011338 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce69283-fdc4-4fc0-bbc7-0a79e776e0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:52.032812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-cce69283-fdc4-4fc0-bbc7-0a79e776e0dd None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:52.034167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-cce69283-fdc4-4fc0-bbc7-0a79e776e0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:52.048610 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b132a0-ba7f-4509-ba64-c9053d317569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:52.049397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b132a0-ba7f-4509-ba64-c9053d317569 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:52.049680 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b132a0-ba7f-4509-ba64-c9053d317569 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:52.049958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b132a0-ba7f-4509-ba64-c9053d317569 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:52.050967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b132a0-ba7f-4509-ba64-c9053d317569 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:52:52.051310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b132a0-ba7f-4509-ba64-c9053d317569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:52.053978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b132a0-ba7f-4509-ba64-c9053d317569 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:52.068782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:52.073235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 16/39] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:52:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:52:52.079274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b132a0-ba7f-4509-ba64-c9053d317569 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:52.143533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:52.206512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b132a0-ba7f-4509-ba64-c9053d317569 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:52.240867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 24/40] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:52:52.246465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1a8b2-51ef-455c-8eeb-46262aa0b227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:52.247199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1a8b2-51ef-455c-8eeb-46262aa0b227 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:52.247482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1a8b2-51ef-455c-8eeb-46262aa0b227 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:52.248455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1a8b2-51ef-455c-8eeb-46262aa0b227 None admin] PATH_INFO: `/v3/users/d171ff8c81304d799c869856dff7ee6d` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:52.249998 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1a8b2-51ef-455c-8eeb-46262aa0b227 None admin] RBAC: Authorizing `identity:get_user(user_id=d171ff8c81304d799c869856dff7ee6d)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:52.251496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1a8b2-51ef-455c-8eeb-46262aa0b227 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:52.253265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 17/41] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:52:52 2022] GET /identity/v3/users/d171ff8c81304d799c869856dff7ee6d => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:52:52.258503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce69283-fdc4-4fc0-bbc7-0a79e776e0dd None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:52.362167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce69283-fdc4-4fc0-bbc7-0a79e776e0dd None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:52.371567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1a8b2-51ef-455c-8eeb-46262aa0b227 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:52.399513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20778-ccfa-455e-96b5-3b374b0789a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:52.400153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20778-ccfa-455e-96b5-3b374b0789a1 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:52.400428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20778-ccfa-455e-96b5-3b374b0789a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:52.400703 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20778-ccfa-455e-96b5-3b374b0789a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:52.401821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe20778-ccfa-455e-96b5-3b374b0789a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:52.403308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe20778-ccfa-455e-96b5-3b374b0789a1 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:52.405568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 25/42] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:52:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:52:52.410890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20778-ccfa-455e-96b5-3b374b0789a1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:52.444254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:52.477259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1a8b2-51ef-455c-8eeb-46262aa0b227 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:52.491389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:52.513346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4723f24-2d08-4bf6-99de-5d856e23f8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:52.514218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4723f24-2d08-4bf6-99de-5d856e23f8bc None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:52.514347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4723f24-2d08-4bf6-99de-5d856e23f8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:52.514516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4723f24-2d08-4bf6-99de-5d856e23f8bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:52.515811 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4723f24-2d08-4bf6-99de-5d856e23f8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:52.517276 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4723f24-2d08-4bf6-99de-5d856e23f8bc None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:52.517385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20778-ccfa-455e-96b5-3b374b0789a1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:52.519127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 18/43] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:52:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:52:52.519857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4723f24-2d08-4bf6-99de-5d856e23f8bc None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:52.551621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404a9b5-7918-40b3-bfd0-1bc1174c128a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:52.552227 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404a9b5-7918-40b3-bfd0-1bc1174c128a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:52.552509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404a9b5-7918-40b3-bfd0-1bc1174c128a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:52.552782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404a9b5-7918-40b3-bfd0-1bc1174c128a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:52.553500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404a9b5-7918-40b3-bfd0-1bc1174c128a None admin] RBAC: Adding query filter params (role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.domain.id=default, user.id=d171ff8c81304d799c869856dff7ee6d) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:52:52.553805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404a9b5-7918-40b3-bfd0-1bc1174c128a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:52.555288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404a9b5-7918-40b3-bfd0-1bc1174c128a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:52.562119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 26/44] 10.222.0.58 () {56 vars in 1322 bytes} [Fri Oct 28 18:52:52 2022] GET /identity/v3/role_assignments?user.id=d171ff8c81304d799c869856dff7ee6d&scope.domain.id=default&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:52:52.563015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404a9b5-7918-40b3-bfd0-1bc1174c128a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:52.630336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4723f24-2d08-4bf6-99de-5d856e23f8bc None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:52.668220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:52.672668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befc2e0-ab30-4643-a9fe-cac6ce6a17a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:52.673301 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befc2e0-ab30-4643-a9fe-cac6ce6a17a1 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:52.673847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befc2e0-ab30-4643-a9fe-cac6ce6a17a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:52.674048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befc2e0-ab30-4643-a9fe-cac6ce6a17a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:52.675138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befc2e0-ab30-4643-a9fe-cac6ce6a17a1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:52.676576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befc2e0-ab30-4643-a9fe-cac6ce6a17a1 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:52.677528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404a9b5-7918-40b3-bfd0-1bc1174c128a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:52.693053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 19/45] 10.222.0.58 () {60 vars in 1131 bytes} [Fri Oct 28 18:52:52 2022] POST /identity/v3/roles => generated 223 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:52:52.723384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c377e4-64c4-4818-8808-70d6a3970b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:52.724123 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c377e4-64c4-4818-8808-70d6a3970b56 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:52.724475 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c377e4-64c4-4818-8808-70d6a3970b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:52.724870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c377e4-64c4-4818-8808-70d6a3970b56 None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:52.725629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c377e4-64c4-4818-8808-70d6a3970b56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:52.729834 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c377e4-64c4-4818-8808-70d6a3970b56 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:52.762838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 27/46] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:52:52 2022] POST /identity/v3/projects => generated 312 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:52:54.905141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a30f3f-4842-41c1-971f-81a651f09b7a None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:54.905640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a30f3f-4842-41c1-971f-81a651f09b7a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:54.905723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a30f3f-4842-41c1-971f-81a651f09b7a None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:54.907044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 20/47] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:52:54.912160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d503d-d74b-47a9-a89f-b5a6072397d0 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:54.912548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d503d-d74b-47a9-a89f-b5a6072397d0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:54.912708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d503d-d74b-47a9-a89f-b5a6072397d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:54.930888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-300d503d-d74b-47a9-a89f-b5a6072397d0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:54.932109 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-300d503d-d74b-47a9-a89f-b5a6072397d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:54.995655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 28/48] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:52:55.269070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a30f3f-4842-41c1-971f-81a651f09b7a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:55.311550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:55.368519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a30f3f-4842-41c1-971f-81a651f09b7a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:55.401020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089be459-8e0e-4245-ab3c-98bc1615ed28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:55.401637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089be459-8e0e-4245-ab3c-98bc1615ed28 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:55.401916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089be459-8e0e-4245-ab3c-98bc1615ed28 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:55.402186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089be459-8e0e-4245-ab3c-98bc1615ed28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:55.403298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089be459-8e0e-4245-ab3c-98bc1615ed28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:55.405248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089be459-8e0e-4245-ab3c-98bc1615ed28 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:55.406776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 21/49] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:52:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:52:55.412395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d503d-d74b-47a9-a89f-b5a6072397d0 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:55.485424 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:55.507878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d503d-d74b-47a9-a89f-b5a6072397d0 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:55.540363 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbe73d-7fe9-487d-82f0-dcf8039f6cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:55.540901 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbe73d-7fe9-487d-82f0-dcf8039f6cbd None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:55.541179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbe73d-7fe9-487d-82f0-dcf8039f6cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:55.541449 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbe73d-7fe9-487d-82f0-dcf8039f6cbd None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:55.542094 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbe73d-7fe9-487d-82f0-dcf8039f6cbd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:55.543566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbe73d-7fe9-487d-82f0-dcf8039f6cbd None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:55.568984 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 29/50] 10.222.0.58 () {60 vars in 1140 bytes} [Fri Oct 28 18:52:55 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:52:57.705747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56e3b7-6c89-4dba-8afc-ea14df04dacb None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:57.706270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56e3b7-6c89-4dba-8afc-ea14df04dacb None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:57.706270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56e3b7-6c89-4dba-8afc-ea14df04dacb None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:57.707678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 22/51] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:52:57.712883 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31108b7-68dd-49f0-9cff-bd84ee0289a7 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:57.713246 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31108b7-68dd-49f0-9cff-bd84ee0289a7 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:57.713444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31108b7-68dd-49f0-9cff-bd84ee0289a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:57.732639 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d31108b7-68dd-49f0-9cff-bd84ee0289a7 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:52:57.733840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d31108b7-68dd-49f0-9cff-bd84ee0289a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:52:57.798804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 30/52] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:52:58.066001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56e3b7-6c89-4dba-8afc-ea14df04dacb None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:58.163698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56e3b7-6c89-4dba-8afc-ea14df04dacb None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:58.173523 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:58.198023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf3b91-bc78-4b17-81a3-07945030efe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:58.198645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf3b91-bc78-4b17-81a3-07945030efe0 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:58.198925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf3b91-bc78-4b17-81a3-07945030efe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:58.199249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf3b91-bc78-4b17-81a3-07945030efe0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:58.200347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf3b91-bc78-4b17-81a3-07945030efe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:58.201798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf3b91-bc78-4b17-81a3-07945030efe0 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:58.203357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 23/53] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:52:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:52:58.208986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31108b7-68dd-49f0-9cff-bd84ee0289a7 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:52:58.264794 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:52:58.305735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31108b7-68dd-49f0-9cff-bd84ee0289a7 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:52:58.337526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b16bea-c5f4-4d0b-ad79-f3c9d5fadaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:52:58.338064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b16bea-c5f4-4d0b-ad79-f3c9d5fadaae None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:52:58.338345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b16bea-c5f4-4d0b-ad79-f3c9d5fadaae None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:52:58.338614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b16bea-c5f4-4d0b-ad79-f3c9d5fadaae None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:52:58.339284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b16bea-c5f4-4d0b-ad79-f3c9d5fadaae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:52:58.340723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b16bea-c5f4-4d0b-ad79-f3c9d5fadaae None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:52:58.367969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 31/54] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:52:58 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:01.597720 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292a216-89f4-4ca5-bb88-2f0c9de964b3 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:01.598608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292a216-89f4-4ca5-bb88-2f0c9de964b3 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:01.599057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292a216-89f4-4ca5-bb88-2f0c9de964b3 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:01.600546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 24/55] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:01.616527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4f2c9-83b1-4837-a99f-29bb88b6f100 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:01.617413 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4f2c9-83b1-4837-a99f-29bb88b6f100 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:01.617925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4f2c9-83b1-4837-a99f-29bb88b6f100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:01.648018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f9e4f2c9-83b1-4837-a99f-29bb88b6f100 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:01.649698 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f9e4f2c9-83b1-4837-a99f-29bb88b6f100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:01.741694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 32/56] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:01.872557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a12c85-01eb-45de-ada1-57bcf3c309ff None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:01.873242 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a12c85-01eb-45de-ada1-57bcf3c309ff None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:01.874275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a12c85-01eb-45de-ada1-57bcf3c309ff None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:01.876096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 25/57] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:01.881384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461b7cd-589a-475e-966d-700dc50975e5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:01.881675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461b7cd-589a-475e-966d-700dc50975e5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:01.881953 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461b7cd-589a-475e-966d-700dc50975e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:01.906493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a461b7cd-589a-475e-966d-700dc50975e5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:01.907090 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce072d33-5de7-4a87-9b8f-d3c2c8dcebc9 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:01.907598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce072d33-5de7-4a87-9b8f-d3c2c8dcebc9 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:01.907699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a461b7cd-589a-475e-966d-700dc50975e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:01.908173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce072d33-5de7-4a87-9b8f-d3c2c8dcebc9 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:01.909721 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 26/58] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:01.927606 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e26ff6-873c-4747-8327-27d9f75fa2ca None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:01.928193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e26ff6-873c-4747-8327-27d9f75fa2ca None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:01.928705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e26ff6-873c-4747-8327-27d9f75fa2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:01.961733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:02.016742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 33/59] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:02.096003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f8e26ff6-873c-4747-8327-27d9f75fa2ca None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:02.097544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f8e26ff6-873c-4747-8327-27d9f75fa2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:02.113444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461b7cd-589a-475e-966d-700dc50975e5 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:02.171724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 27/60] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:02.179010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab340f98-3223-4dce-8db6-ebe0f6c24df0 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.179010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab340f98-3223-4dce-8db6-ebe0f6c24df0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.179010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab340f98-3223-4dce-8db6-ebe0f6c24df0 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.179010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 28/61] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:02.195423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd15580-ca9b-475a-9b0f-dda388731901 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.195705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd15580-ca9b-475a-9b0f-dda388731901 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.195968 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd15580-ca9b-475a-9b0f-dda388731901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.231592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-bcd15580-ca9b-475a-9b0f-dda388731901 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:02.232906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-bcd15580-ca9b-475a-9b0f-dda388731901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:02.310675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461b7cd-589a-475e-966d-700dc50975e5 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:02.313950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 29/62] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:02.324113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd15580-ca9b-475a-9b0f-dda388731901 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:02.383981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80d54e-5efb-4222-ac86-fc2ce5b721cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:02.385586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:02.386283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80d54e-5efb-4222-ac86-fc2ce5b721cc None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.386498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80d54e-5efb-4222-ac86-fc2ce5b721cc None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.386780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80d54e-5efb-4222-ac86-fc2ce5b721cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.401264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80d54e-5efb-4222-ac86-fc2ce5b721cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-0c80d54e-5efb-4222-ac86-fc2ce5b721cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:02.407807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:02.408656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:02.409416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:02.409984 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 34/63] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:02.411594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe8b29-c802-4974-83a1-b139b5dee64d None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.411876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe8b29-c802-4974-83a1-b139b5dee64d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.412151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe8b29-c802-4974-83a1-b139b5dee64d None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.415058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 35/64] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:02.419067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe8b29-c802-4974-83a1-b139b5dee64d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:02.456874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:02.476911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd15580-ca9b-475a-9b0f-dda388731901 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:02.513486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414b3fb-c01f-46e6-84fa-7a128f437fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:02.514658 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b3fb-c01f-46e6-84fa-7a128f437fdb None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.515119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b3fb-c01f-46e6-84fa-7a128f437fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.515392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b3fb-c01f-46e6-84fa-7a128f437fdb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.525401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe8b29-c802-4974-83a1-b139b5dee64d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:02.526341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414b3fb-c01f-46e6-84fa-7a128f437fdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-b414b3fb-c01f-46e6-84fa-7a128f437fdb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:02.532068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:02.532897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:02.533773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 30/65] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:53:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:02.534908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d5371-c780-4c99-9aed-48451577baf8 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.535101 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d5371-c780-4c99-9aed-48451577baf8 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.535367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d5371-c780-4c99-9aed-48451577baf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.553561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-812d5371-c780-4c99-9aed-48451577baf8 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:02.554800 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-812d5371-c780-4c99-9aed-48451577baf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:02.564267 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfcf50-e788-46f7-9696-444ae6832c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:02.564886 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfcf50-e788-46f7-9696-444ae6832c4a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.565172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfcf50-e788-46f7-9696-444ae6832c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.565450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfcf50-e788-46f7-9696-444ae6832c4a None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.566126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfcf50-e788-46f7-9696-444ae6832c4a None admin] RBAC: Adding query filter params (name=member) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:02.566441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfcf50-e788-46f7-9696-444ae6832c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:02.567889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfcf50-e788-46f7-9696-444ae6832c4a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:02.580854 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 36/66] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:02.580854 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfcf50-e788-46f7-9696-444ae6832c4a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:02.628534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 31/67] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:02.629167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d5371-c780-4c99-9aed-48451577baf8 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:02.663909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:02.679899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:02.687756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfcf50-e788-46f7-9696-444ae6832c4a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:02.724187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733a92-1306-4c60-9ef2-c2dda14cdfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:02.724739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733a92-1306-4c60-9ef2-c2dda14cdfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.725018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733a92-1306-4c60-9ef2-c2dda14cdfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.725289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733a92-1306-4c60-9ef2-c2dda14cdfe6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.736017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09733a92-1306-4c60-9ef2-c2dda14cdfe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-09733a92-1306-4c60-9ef2-c2dda14cdfe6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:02.741062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:02.741902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:02.747810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:02.749082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d5371-c780-4c99-9aed-48451577baf8 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:02.749176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 37/68] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:53:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:02.749176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733a92-1306-4c60-9ef2-c2dda14cdfe6 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:02.793434 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce83632-a647-4b98-826e-ffff68a48e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:02.794142 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce83632-a647-4b98-826e-ffff68a48e8d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.794475 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce83632-a647-4b98-826e-ffff68a48e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.794754 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce83632-a647-4b98-826e-ffff68a48e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.795505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce83632-a647-4b98-826e-ffff68a48e8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:02.796153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce83632-a647-4b98-826e-ffff68a48e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:02.797934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce83632-a647-4b98-826e-ffff68a48e8d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:02.813032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 32/69] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:02.813032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce83632-a647-4b98-826e-ffff68a48e8d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:02.855179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733a92-1306-4c60-9ef2-c2dda14cdfe6 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:02.889910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d74f30-bdc7-4c6f-aed6-1620c8732af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:02.890666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d74f30-bdc7-4c6f-aed6-1620c8732af3 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.891052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d74f30-bdc7-4c6f-aed6-1620c8732af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.891401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d74f30-bdc7-4c6f-aed6-1620c8732af3 None admin] PATH_INFO: `/v3/users/27e08ff6bb314f25a262830c9d8000e2` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.905920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d74f30-bdc7-4c6f-aed6-1620c8732af3 None admin] RBAC: Authorizing `identity:get_user(user_id=27e08ff6bb314f25a262830c9d8000e2)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:02.906446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:02.907450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d74f30-bdc7-4c6f-aed6-1620c8732af3 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:02.909251 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 38/70] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:02 2022] GET /identity/v3/users/27e08ff6bb314f25a262830c9d8000e2 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:02.909866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d74f30-bdc7-4c6f-aed6-1620c8732af3 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:02.924376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce83632-a647-4b98-826e-ffff68a48e8d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:02.925084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:02.958774 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7832d2-fd84-483c-ac4a-22e41b9306bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:02.959440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7832d2-fd84-483c-ac4a-22e41b9306bf None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:02.959723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7832d2-fd84-483c-ac4a-22e41b9306bf None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:02.959995 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7832d2-fd84-483c-ac4a-22e41b9306bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:02.970432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7832d2-fd84-483c-ac4a-22e41b9306bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-8d7832d2-fd84-483c-ac4a-22e41b9306bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:02.975899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:02.976908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:02.977673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 33/71] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:02.978304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7832d2-fd84-483c-ac4a-22e41b9306bf None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.012112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d74f30-bdc7-4c6f-aed6-1620c8732af3 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.045243 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd6ca-c356-4dd6-bc54-b6a87214e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.046459 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd6ca-c356-4dd6-bc54-b6a87214e1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.046736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd6ca-c356-4dd6-bc54-b6a87214e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.047027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd6ca-c356-4dd6-bc54-b6a87214e1b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.047686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd6ca-c356-4dd6-bc54-b6a87214e1b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:03.047985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd6ca-c356-4dd6-bc54-b6a87214e1b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.049354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd6ca-c356-4dd6-bc54-b6a87214e1b5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.057047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 39/72] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:03.057818 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd6ca-c356-4dd6-bc54-b6a87214e1b5 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.077863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7832d2-fd84-483c-ac4a-22e41b9306bf None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.112005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cbf56-679a-4272-9f0a-421bdcaa265d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.112638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cbf56-679a-4272-9f0a-421bdcaa265d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.112930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cbf56-679a-4272-9f0a-421bdcaa265d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.113207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cbf56-679a-4272-9f0a-421bdcaa265d None admin] PATH_INFO: `/v3/users/27e08ff6bb314f25a262830c9d8000e2` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.114350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cbf56-679a-4272-9f0a-421bdcaa265d None admin] RBAC: Authorizing `identity:get_user(user_id=27e08ff6bb314f25a262830c9d8000e2)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.115820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cbf56-679a-4272-9f0a-421bdcaa265d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.117377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 34/73] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:02 2022] GET /identity/v3/users/27e08ff6bb314f25a262830c9d8000e2 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:03.118134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cbf56-679a-4272-9f0a-421bdcaa265d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.135426 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:03.140089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:03.164071 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd6ca-c356-4dd6-bc54-b6a87214e1b5 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.195814 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47764872-667a-4d28-bfef-255956c2014d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.196417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47764872-667a-4d28-bfef-255956c2014d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.196695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47764872-667a-4d28-bfef-255956c2014d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.196967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47764872-667a-4d28-bfef-255956c2014d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.198061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47764872-667a-4d28-bfef-255956c2014d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.199544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47764872-667a-4d28-bfef-255956c2014d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.201098 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 40/74] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:03.202428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47764872-667a-4d28-bfef-255956c2014d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.219812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cbf56-679a-4272-9f0a-421bdcaa265d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.252484 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8de18-ad9e-4951-b9a3-ec9988552b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.253046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8de18-ad9e-4951-b9a3-ec9988552b8d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.253326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8de18-ad9e-4951-b9a3-ec9988552b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.253596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8de18-ad9e-4951-b9a3-ec9988552b8d None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.254706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8de18-ad9e-4951-b9a3-ec9988552b8d None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.256164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8de18-ad9e-4951-b9a3-ec9988552b8d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.257741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 35/75] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:03 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:03.262526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8de18-ad9e-4951-b9a3-ec9988552b8d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.306668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47764872-667a-4d28-bfef-255956c2014d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.339380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c8c0-05e4-4c91-baab-44231461074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.339945 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:03.340056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c8c0-05e4-4c91-baab-44231461074f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.340245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c8c0-05e4-4c91-baab-44231461074f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.340513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c8c0-05e4-4c91-baab-44231461074f None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.341169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224c8c0-05e4-4c91-baab-44231461074f None admin] RBAC: Adding query filter params (name=member) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:03.341479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224c8c0-05e4-4c91-baab-44231461074f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.343584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224c8c0-05e4-4c91-baab-44231461074f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.351395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 41/76] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:03.352172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c8c0-05e4-4c91-baab-44231461074f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.357733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8de18-ad9e-4951-b9a3-ec9988552b8d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.360451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:03.392832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e659e7-6acd-4acc-bbdd-2a1827a3087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.393717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e659e7-6acd-4acc-bbdd-2a1827a3087c None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.393999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e659e7-6acd-4acc-bbdd-2a1827a3087c None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.394536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e659e7-6acd-4acc-bbdd-2a1827a3087c None admin] PATH_INFO: `/v3/users/d171ff8c81304d799c869856dff7ee6d` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.395477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e659e7-6acd-4acc-bbdd-2a1827a3087c None admin] RBAC: Authorizing `identity:get_user(user_id=d171ff8c81304d799c869856dff7ee6d)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.396940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e659e7-6acd-4acc-bbdd-2a1827a3087c None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.398493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 36/77] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:03 2022] GET /identity/v3/users/d171ff8c81304d799c869856dff7ee6d => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:03.403746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e659e7-6acd-4acc-bbdd-2a1827a3087c None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.450060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c8c0-05e4-4c91-baab-44231461074f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.482653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa57406-1af6-4b13-80a2-e9e091a3e98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.483407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa57406-1af6-4b13-80a2-e9e091a3e98f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.483687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa57406-1af6-4b13-80a2-e9e091a3e98f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.483958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa57406-1af6-4b13-80a2-e9e091a3e98f None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.485063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa57406-1af6-4b13-80a2-e9e091a3e98f None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.486493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa57406-1af6-4b13-80a2-e9e091a3e98f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.488136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 42/78] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:03 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:03.491835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa57406-1af6-4b13-80a2-e9e091a3e98f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.500923 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e659e7-6acd-4acc-bbdd-2a1827a3087c None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.535330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395adab-1f42-4d14-9b8f-3c920c38974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.535932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395adab-1f42-4d14-9b8f-3c920c38974a None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.536212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395adab-1f42-4d14-9b8f-3c920c38974a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.536513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395adab-1f42-4d14-9b8f-3c920c38974a None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.537149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395adab-1f42-4d14-9b8f-3c920c38974a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.538583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395adab-1f42-4d14-9b8f-3c920c38974a None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.566455 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:03.566995 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 37/79] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:03 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:03.567945 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395adab-1f42-4d14-9b8f-3c920c38974a None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.589407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:03.599846 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa57406-1af6-4b13-80a2-e9e091a3e98f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.646627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aac40-fd08-42e2-ac6e-5cd3f339ba45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.647716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aac40-fd08-42e2-ac6e-5cd3f339ba45 None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.648330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aac40-fd08-42e2-ac6e-5cd3f339ba45 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.648610 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aac40-fd08-42e2-ac6e-5cd3f339ba45 None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652/users/27e08ff6bb314f25a262830c9d8000e2/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.650792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91aac40-fd08-42e2-ac6e-5cd3f339ba45 None admin] RBAC: Authorizing `identity:create_grant(project_id=96c771455a484eb080ab85c17582f652, user_id=27e08ff6bb314f25a262830c9d8000e2, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.652876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91aac40-fd08-42e2-ac6e-5cd3f339ba45 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.679018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 43/80] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:03 2022] PUT /identity/v3/projects/96c771455a484eb080ab85c17582f652/users/27e08ff6bb314f25a262830c9d8000e2/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:03.679018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aac40-fd08-42e2-ac6e-5cd3f339ba45 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.689636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395adab-1f42-4d14-9b8f-3c920c38974a None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.740003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e9ee6-6c90-40ec-89d6-744d5ecbcb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.740669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e9ee6-6c90-40ec-89d6-744d5ecbcb2a None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.740948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e9ee6-6c90-40ec-89d6-744d5ecbcb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.741223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e9ee6-6c90-40ec-89d6-744d5ecbcb2a None admin] PATH_INFO: `/v3/users/27e08ff6bb314f25a262830c9d8000e2` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.742463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e9ee6-6c90-40ec-89d6-744d5ecbcb2a None admin] RBAC: Authorizing `identity:get_user(user_id=27e08ff6bb314f25a262830c9d8000e2)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.744020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e9ee6-6c90-40ec-89d6-744d5ecbcb2a None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.746724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 38/81] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:03 2022] GET /identity/v3/users/27e08ff6bb314f25a262830c9d8000e2 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:03.747673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e9ee6-6c90-40ec-89d6-744d5ecbcb2a None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.841562 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aac40-fd08-42e2-ac6e-5cd3f339ba45 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.856090 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:03.887708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e9ee6-6c90-40ec-89d6-744d5ecbcb2a None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:03.891867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2e6dc-08d6-4d21-bf3b-937702c8dcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.892520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2e6dc-08d6-4d21-bf3b-937702c8dcfe None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.892808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2e6dc-08d6-4d21-bf3b-937702c8dcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.893352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2e6dc-08d6-4d21-bf3b-937702c8dcfe None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.894515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2e6dc-08d6-4d21-bf3b-937702c8dcfe None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.896001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2e6dc-08d6-4d21-bf3b-937702c8dcfe None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.897672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 44/82] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:03 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:03.898449 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2e6dc-08d6-4d21-bf3b-937702c8dcfe None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:03.901608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:03.939071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2882f-73bf-4964-8ef8-9fd13a2e3bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:03.939874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2882f-73bf-4964-8ef8-9fd13a2e3bb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:03.940159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2882f-73bf-4964-8ef8-9fd13a2e3bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:03.940527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2882f-73bf-4964-8ef8-9fd13a2e3bb7 None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652/users/27e08ff6bb314f25a262830c9d8000e2/roles/0481df198a1a42f0a00358e0b9a938c5` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:03.943764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2882f-73bf-4964-8ef8-9fd13a2e3bb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=96c771455a484eb080ab85c17582f652, user_id=27e08ff6bb314f25a262830c9d8000e2, role_id=0481df198a1a42f0a00358e0b9a938c5)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:03.945246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2882f-73bf-4964-8ef8-9fd13a2e3bb7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:03.964921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 39/83] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:03 2022] PUT /identity/v3/projects/96c771455a484eb080ab85c17582f652/users/27e08ff6bb314f25a262830c9d8000e2/roles/0481df198a1a42f0a00358e0b9a938c5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:03.965800 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2882f-73bf-4964-8ef8-9fd13a2e3bb7 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:04.068329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2e6dc-08d6-4d21-bf3b-937702c8dcfe None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:04.084054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2882f-73bf-4964-8ef8-9fd13a2e3bb7 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:04.122835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9477a4-6096-4c55-aef8-87f76541168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:04.123522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9477a4-6096-4c55-aef8-87f76541168c None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:04.123814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9477a4-6096-4c55-aef8-87f76541168c None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:04.124097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9477a4-6096-4c55-aef8-87f76541168c None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652/users/d171ff8c81304d799c869856dff7ee6d/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:04.126175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9477a4-6096-4c55-aef8-87f76541168c None admin] RBAC: Authorizing `identity:create_grant(project_id=96c771455a484eb080ab85c17582f652, user_id=d171ff8c81304d799c869856dff7ee6d, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:04.127644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954b525-f97f-4c89-b2ba-1ce6ff4d6e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:04.127772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9477a4-6096-4c55-aef8-87f76541168c None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:04.128287 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954b525-f97f-4c89-b2ba-1ce6ff4d6e6d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:04.128567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954b525-f97f-4c89-b2ba-1ce6ff4d6e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:04.128837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954b525-f97f-4c89-b2ba-1ce6ff4d6e6d None admin] PATH_INFO: `/v3/projects/c454bdb8429b4f7ab29e6a38dd69a4b3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:04.133708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:04.135286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954b525-f97f-4c89-b2ba-1ce6ff4d6e6d None admin] RBAC: Authorizing `identity:get_project(project_id=c454bdb8429b4f7ab29e6a38dd69a4b3)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:04.136936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954b525-f97f-4c89-b2ba-1ce6ff4d6e6d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:04.138772 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 45/84] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:03 2022] GET /identity/v3/projects/c454bdb8429b4f7ab29e6a38dd69a4b3 => generated 312 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:04.143510 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 40/85] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:03 2022] PUT /identity/v3/projects/96c771455a484eb080ab85c17582f652/users/d171ff8c81304d799c869856dff7ee6d/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:04.153644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954b525-f97f-4c89-b2ba-1ce6ff4d6e6d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:04.245409 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:04.285589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954b525-f97f-4c89-b2ba-1ce6ff4d6e6d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:04.332756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980894ad-74f5-4978-87a3-157da43c0e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:04.333483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980894ad-74f5-4978-87a3-157da43c0e1b None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:04.333730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980894ad-74f5-4978-87a3-157da43c0e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:04.333996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980894ad-74f5-4978-87a3-157da43c0e1b None admin] PATH_INFO: `/v3/projects/c454bdb8429b4f7ab29e6a38dd69a4b3/users/27e08ff6bb314f25a262830c9d8000e2/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:04.336038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980894ad-74f5-4978-87a3-157da43c0e1b None admin] RBAC: Authorizing `identity:create_grant(project_id=c454bdb8429b4f7ab29e6a38dd69a4b3, user_id=27e08ff6bb314f25a262830c9d8000e2, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:04.337603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980894ad-74f5-4978-87a3-157da43c0e1b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:04.366857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 46/86] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:04 2022] PUT /identity/v3/projects/c454bdb8429b4f7ab29e6a38dd69a4b3/users/27e08ff6bb314f25a262830c9d8000e2/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:06.282633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d6ed7-0a2d-4422-91fd-24311977a1c0 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:06.283179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d6ed7-0a2d-4422-91fd-24311977a1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:06.283179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d6ed7-0a2d-4422-91fd-24311977a1c0 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:06.284685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 41/87] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:06.294562 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f9d56-9a6a-4557-9629-019b4c5f2e6e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:06.295009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f9d56-9a6a-4557-9629-019b4c5f2e6e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:06.295190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f9d56-9a6a-4557-9629-019b4c5f2e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:06.324879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6b7f9d56-9a6a-4557-9629-019b4c5f2e6e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:06.326089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6b7f9d56-9a6a-4557-9629-019b4c5f2e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:06.426564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 47/88] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:06.967883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d6ed7-0a2d-4422-91fd-24311977a1c0 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:07.035421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214986da-b6c9-4d4f-8e3a-4d5bf377cc69 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.035739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214986da-b6c9-4d4f-8e3a-4d5bf377cc69 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.035967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214986da-b6c9-4d4f-8e3a-4d5bf377cc69 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.037319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 48/89] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:07.052769 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e7c9f-d7b3-45fe-b843-ac3e8714e49f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.053052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e7c9f-d7b3-45fe-b843-ac3e8714e49f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.053307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e7c9f-d7b3-45fe-b843-ac3e8714e49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.080701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-5c3e7c9f-d7b3-45fe-b843-ac3e8714e49f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:07.081885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5c3e7c9f-d7b3-45fe-b843-ac3e8714e49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:07.124358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d6ed7-0a2d-4422-91fd-24311977a1c0 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:07.189397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bb463-c549-4deb-aeba-3016c97ef944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:07.190038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb463-c549-4deb-aeba-3016c97ef944 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.190319 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb463-c549-4deb-aeba-3016c97ef944 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.190587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb463-c549-4deb-aeba-3016c97ef944 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.208611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:07.209750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bb463-c549-4deb-aeba-3016c97ef944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-f48bb463-c549-4deb-aeba-3016c97ef944 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:07.215474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:07.216438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:07.217222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 42/90] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:07.228388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bb463-c549-4deb-aeba-3016c97ef944 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:07.244685 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 49/91] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:07.261634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb66c09-16c1-4745-b861-a974f6098465 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.261954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb66c09-16c1-4745-b861-a974f6098465 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.262208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb66c09-16c1-4745-b861-a974f6098465 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.263602 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 50/92] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:07.268941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758bc78-98e1-4145-bec5-17ad93235262 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.269225 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758bc78-98e1-4145-bec5-17ad93235262 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.269498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758bc78-98e1-4145-bec5-17ad93235262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.304138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-e758bc78-98e1-4145-bec5-17ad93235262 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:07.305278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e758bc78-98e1-4145-bec5-17ad93235262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:07.335369 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:07.395634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bb463-c549-4deb-aeba-3016c97ef944 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:07.419853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 51/93] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:07.422660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ade11e-3d1a-4c50-9785-3549bcfcad27 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.422947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ade11e-3d1a-4c50-9785-3549bcfcad27 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.423248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ade11e-3d1a-4c50-9785-3549bcfcad27 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.425595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 52/94] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:07.426912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d6530-17e8-45e0-8208-e0d33b940d8f None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.427609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d6530-17e8-45e0-8208-e0d33b940d8f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.427888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d6530-17e8-45e0-8208-e0d33b940d8f None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.429686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 53/95] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:07.430707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1935b14-337e-48dd-a595-4af19a0d16c7 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.432644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccf1dd-fbc7-4403-9816-b3384571ba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:07.433264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf1dd-fbc7-4403-9816-b3384571ba7a None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.433547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf1dd-fbc7-4403-9816-b3384571ba7a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.433825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf1dd-fbc7-4403-9816-b3384571ba7a None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.434495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccf1dd-fbc7-4403-9816-b3384571ba7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:07.434916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccf1dd-fbc7-4403-9816-b3384571ba7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:07.435043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1935b14-337e-48dd-a595-4af19a0d16c7 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.435257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1935b14-337e-48dd-a595-4af19a0d16c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.436343 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccf1dd-fbc7-4403-9816-b3384571ba7a None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:07.456929 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 43/96] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:07.458157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147eb57-1c15-403d-8a4c-f5b1cbf70d5f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.458442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147eb57-1c15-403d-8a4c-f5b1cbf70d5f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.458717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147eb57-1c15-403d-8a4c-f5b1cbf70d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.470765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a1935b14-337e-48dd-a595-4af19a0d16c7 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:07.479019 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a1935b14-337e-48dd-a595-4af19a0d16c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:07.495994 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-4147eb57-1c15-403d-8a4c-f5b1cbf70d5f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:07.497175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4147eb57-1c15-403d-8a4c-f5b1cbf70d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:07.585829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 54/97] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:07.587375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1935b14-337e-48dd-a595-4af19a0d16c7 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:07.591805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 44/98] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:07.593033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147eb57-1c15-403d-8a4c-f5b1cbf70d5f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:07.667342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:07.691400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:07.755554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1935b14-337e-48dd-a595-4af19a0d16c7 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:07.775252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147eb57-1c15-403d-8a4c-f5b1cbf70d5f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:07.820885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e063270-6923-46d8-9f9c-599390845830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:07.823041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e063270-6923-46d8-9f9c-599390845830 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.823695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e063270-6923-46d8-9f9c-599390845830 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.824717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e063270-6923-46d8-9f9c-599390845830 None admin] PATH_INFO: `/v3/users/80c6e88527a243b2970a826e063aeecc` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.834767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc32b5-5807-4d02-877d-f3862a587603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:07.835754 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc32b5-5807-4d02-877d-f3862a587603 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:07.836302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc32b5-5807-4d02-877d-f3862a587603 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:07.836779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc32b5-5807-4d02-877d-f3862a587603 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:07.838082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc32b5-5807-4d02-877d-f3862a587603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:07.839838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc32b5-5807-4d02-877d-f3862a587603 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:07.842685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 45/99] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:07.848553 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc32b5-5807-4d02-877d-f3862a587603 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:07.852178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e063270-6923-46d8-9f9c-599390845830 None admin] RBAC: Authorizing `identity:get_user(user_id=80c6e88527a243b2970a826e063aeecc)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:07.853643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e063270-6923-46d8-9f9c-599390845830 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:07.855615 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 55/100] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:07 2022] GET /identity/v3/users/80c6e88527a243b2970a826e063aeecc => generated 289 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:07.856364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e063270-6923-46d8-9f9c-599390845830 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:07.984304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc32b5-5807-4d02-877d-f3862a587603 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:07.997419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e063270-6923-46d8-9f9c-599390845830 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.004809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:08.010386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:08.027613 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795a40e-140c-430e-86b8-a7863f0d5a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.028255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795a40e-140c-430e-86b8-a7863f0d5a36 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.028533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795a40e-140c-430e-86b8-a7863f0d5a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.028806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795a40e-140c-430e-86b8-a7863f0d5a36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.038287 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e458c4-4b83-4047-9162-394e87e6213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.038891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e458c4-4b83-4047-9162-394e87e6213f None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.039264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e458c4-4b83-4047-9162-394e87e6213f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.039537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e458c4-4b83-4047-9162-394e87e6213f None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.039632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795a40e-140c-430e-86b8-a7863f0d5a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:08.040182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e458c4-4b83-4047-9162-394e87e6213f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:08.041610 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e458c4-4b83-4047-9162-394e87e6213f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-5795a40e-140c-430e-86b8-a7863f0d5a36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:08.045001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:08.045865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:08.046708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 46/101] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:53:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:08.047776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795a40e-140c-430e-86b8-a7863f0d5a36 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.079657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 56/102] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:53:07 2022] POST /identity/v3/projects => generated 302 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:08.081016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e458c4-4b83-4047-9162-394e87e6213f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.171009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795a40e-140c-430e-86b8-a7863f0d5a36 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.201033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e458c4-4b83-4047-9162-394e87e6213f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.209625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311efc5-060b-4180-a3cc-0b67ded6184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.210572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311efc5-060b-4180-a3cc-0b67ded6184d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.210859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311efc5-060b-4180-a3cc-0b67ded6184d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.211178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311efc5-060b-4180-a3cc-0b67ded6184d None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.212440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311efc5-060b-4180-a3cc-0b67ded6184d None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:08.213956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311efc5-060b-4180-a3cc-0b67ded6184d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:08.215722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 47/103] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:08 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:08.220861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311efc5-060b-4180-a3cc-0b67ded6184d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.243776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce456541-7686-4549-9564-d134162782e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.244421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce456541-7686-4549-9564-d134162782e0 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.244704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce456541-7686-4549-9564-d134162782e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.244980 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce456541-7686-4549-9564-d134162782e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.257120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce456541-7686-4549-9564-d134162782e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-ce456541-7686-4549-9564-d134162782e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:08.262507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:08.263445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:08.264987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:08.267093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 57/104] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:53:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:08.269000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce456541-7686-4549-9564-d134162782e0 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.274705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:08.294534 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:08.337077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311efc5-060b-4180-a3cc-0b67ded6184d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.376974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9345350-ab8f-4aec-a093-8afd404ce937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.377932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9345350-ab8f-4aec-a093-8afd404ce937 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.378217 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9345350-ab8f-4aec-a093-8afd404ce937 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.378748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9345350-ab8f-4aec-a093-8afd404ce937 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.387938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce456541-7686-4549-9564-d134162782e0 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.392144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9345350-ab8f-4aec-a093-8afd404ce937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-c9345350-ab8f-4aec-a093-8afd404ce937 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:08.399526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:08.400490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:08.401258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:08.406541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 48/105] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:08.406541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7161-7099-4194-b6ed-66057f461fe9 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.406541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7161-7099-4194-b6ed-66057f461fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.406541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7161-7099-4194-b6ed-66057f461fe9 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.406541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 49/106] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:08.407815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7161-7099-4194-b6ed-66057f461fe9 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.429881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8bc9e-70fb-4070-87ff-b8cd1ba4adc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.430542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8bc9e-70fb-4070-87ff-b8cd1ba4adc3 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.430829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8bc9e-70fb-4070-87ff-b8cd1ba4adc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.431138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8bc9e-70fb-4070-87ff-b8cd1ba4adc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.431876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8bc9e-70fb-4070-87ff-b8cd1ba4adc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:08.432117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8bc9e-70fb-4070-87ff-b8cd1ba4adc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:08.433509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8bc9e-70fb-4070-87ff-b8cd1ba4adc3 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:08.443274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 58/107] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:08.447175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8bc9e-70fb-4070-87ff-b8cd1ba4adc3 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.523162 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7161-7099-4194-b6ed-66057f461fe9 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.525409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:08.533485 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:08.554055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8bc9e-70fb-4070-87ff-b8cd1ba4adc3 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.560059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a81999-e961-41f6-bc51-cb32590f20df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.560733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a81999-e961-41f6-bc51-cb32590f20df None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.561570 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a81999-e961-41f6-bc51-cb32590f20df None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.561846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a81999-e961-41f6-bc51-cb32590f20df None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652/users/80c6e88527a243b2970a826e063aeecc/roles/5981233f19954c69b900f0bbdbb84fd0` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.564161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a81999-e961-41f6-bc51-cb32590f20df None admin] RBAC: Authorizing `identity:create_grant(project_id=96c771455a484eb080ab85c17582f652, user_id=80c6e88527a243b2970a826e063aeecc, role_id=5981233f19954c69b900f0bbdbb84fd0)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:08.566262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a81999-e961-41f6-bc51-cb32590f20df None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:08.579851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 50/108] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:08 2022] PUT /identity/v3/projects/96c771455a484eb080ab85c17582f652/users/80c6e88527a243b2970a826e063aeecc/roles/5981233f19954c69b900f0bbdbb84fd0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:08.580782 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f9eda-8085-4f0d-9f89-0678a1de0370 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.581063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f9eda-8085-4f0d-9f89-0678a1de0370 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.581333 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f9eda-8085-4f0d-9f89-0678a1de0370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.601177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-401f9eda-8085-4f0d-9f89-0678a1de0370 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:08.602415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-401f9eda-8085-4f0d-9f89-0678a1de0370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:08.611427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c60882-2d13-462a-8b75-8b1eb7dcc9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.613101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c60882-2d13-462a-8b75-8b1eb7dcc9af None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.613625 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c60882-2d13-462a-8b75-8b1eb7dcc9af None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.614598 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c60882-2d13-462a-8b75-8b1eb7dcc9af None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.615530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c60882-2d13-462a-8b75-8b1eb7dcc9af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:08.616458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c60882-2d13-462a-8b75-8b1eb7dcc9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:08.618741 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c60882-2d13-462a-8b75-8b1eb7dcc9af None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:08.632043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 59/109] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:08.633820 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c60882-2d13-462a-8b75-8b1eb7dcc9af None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.683074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 51/110] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:08.683640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f9eda-8085-4f0d-9f89-0678a1de0370 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.801831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:08.803340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c60882-2d13-462a-8b75-8b1eb7dcc9af None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.816880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f9eda-8085-4f0d-9f89-0678a1de0370 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.856996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707dd54-1ab1-4781-874f-d584974afcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.857635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707dd54-1ab1-4781-874f-d584974afcd5 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.858476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707dd54-1ab1-4781-874f-d584974afcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.858750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707dd54-1ab1-4781-874f-d584974afcd5 None admin] PATH_INFO: `/v3/users/d171ff8c81304d799c869856dff7ee6d` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.859185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6d06a-66a2-4ee0-a4e5-20b10a8a0fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:08.860053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6d06a-66a2-4ee0-a4e5-20b10a8a0fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:08.860155 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707dd54-1ab1-4781-874f-d584974afcd5 None admin] RBAC: Authorizing `identity:get_user(user_id=d171ff8c81304d799c869856dff7ee6d)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:08.860611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6d06a-66a2-4ee0-a4e5-20b10a8a0fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:08.861057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6d06a-66a2-4ee0-a4e5-20b10a8a0fd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:08.861831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707dd54-1ab1-4781-874f-d584974afcd5 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:08.862086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6d06a-66a2-4ee0-a4e5-20b10a8a0fd5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:08.862557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6d06a-66a2-4ee0-a4e5-20b10a8a0fd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:08.864160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 52/111] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:08 2022] GET /identity/v3/users/d171ff8c81304d799c869856dff7ee6d => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:08.864808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6d06a-66a2-4ee0-a4e5-20b10a8a0fd5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:08.865308 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707dd54-1ab1-4781-874f-d584974afcd5 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.876112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 60/112] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:08.882053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6d06a-66a2-4ee0-a4e5-20b10a8a0fd5 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:08.893586 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:08.984225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707dd54-1ab1-4781-874f-d584974afcd5 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:08.996441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6d06a-66a2-4ee0-a4e5-20b10a8a0fd5 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:09.021606 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a79135-ac12-49d9-9435-742458da1be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:09.022116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a79135-ac12-49d9-9435-742458da1be1 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:09.022529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a79135-ac12-49d9-9435-742458da1be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:09.022869 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a79135-ac12-49d9-9435-742458da1be1 None admin] PATH_INFO: `/v3/users/27e08ff6bb314f25a262830c9d8000e2` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:09.024258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a79135-ac12-49d9-9435-742458da1be1 None admin] RBAC: Authorizing `identity:get_user(user_id=27e08ff6bb314f25a262830c9d8000e2)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:09.026274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a79135-ac12-49d9-9435-742458da1be1 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:09.028003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 53/113] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:08 2022] GET /identity/v3/users/27e08ff6bb314f25a262830c9d8000e2 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:09.029100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a79135-ac12-49d9-9435-742458da1be1 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:09.035671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cb08f-8501-40ab-82e2-c00394ba2bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:09.036669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cb08f-8501-40ab-82e2-c00394ba2bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:09.037125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cb08f-8501-40ab-82e2-c00394ba2bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:09.037573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cb08f-8501-40ab-82e2-c00394ba2bc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:09.067309 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:09.111222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cb08f-8501-40ab-82e2-c00394ba2bc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:09.113091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cb08f-8501-40ab-82e2-c00394ba2bc6 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:09.121124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:09.146962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a79135-ac12-49d9-9435-742458da1be1 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:09.184285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf340ca-5be9-4725-a5a0-0adab207fc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:09.185223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf340ca-5be9-4725-a5a0-0adab207fc9e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:09.185633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf340ca-5be9-4725-a5a0-0adab207fc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:09.186102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf340ca-5be9-4725-a5a0-0adab207fc9e None admin] PATH_INFO: `/v3/users/27e08ff6bb314f25a262830c9d8000e2` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:09.187392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf340ca-5be9-4725-a5a0-0adab207fc9e None admin] RBAC: Authorizing `identity:get_user(user_id=27e08ff6bb314f25a262830c9d8000e2)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:09.188963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf340ca-5be9-4725-a5a0-0adab207fc9e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:09.190784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 54/114] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:09 2022] GET /identity/v3/users/27e08ff6bb314f25a262830c9d8000e2 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:09.196007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf340ca-5be9-4725-a5a0-0adab207fc9e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:09.217666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 61/115] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:09.218683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cb08f-8501-40ab-82e2-c00394ba2bc6 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:09.303385 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:09.308003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf340ca-5be9-4725-a5a0-0adab207fc9e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:09.319769 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cb08f-8501-40ab-82e2-c00394ba2bc6 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:09.328033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:09.347791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95487050-887b-4c3f-9c66-41557835f140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:09.349020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95487050-887b-4c3f-9c66-41557835f140 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:09.349435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95487050-887b-4c3f-9c66-41557835f140 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:09.349858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95487050-887b-4c3f-9c66-41557835f140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:09.356915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bd108-858d-4794-b662-9eb2e841ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:09.357714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bd108-858d-4794-b662-9eb2e841ee97 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:09.358118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bd108-858d-4794-b662-9eb2e841ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:09.358488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bd108-858d-4794-b662-9eb2e841ee97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:09.370212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bd108-858d-4794-b662-9eb2e841ee97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-db4bd108-858d-4794-b662-9eb2e841ee97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:09.375453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:09.376306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:09.377039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 62/116] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:09.377657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bd108-858d-4794-b662-9eb2e841ee97 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:09.428071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95487050-887b-4c3f-9c66-41557835f140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:09.429992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95487050-887b-4c3f-9c66-41557835f140 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:09.481340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bd108-858d-4794-b662-9eb2e841ee97 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:09.516707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea016866-d7fb-4ee0-90f4-c3957deee01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:09.517483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea016866-d7fb-4ee0-90f4-c3957deee01a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:09.517863 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea016866-d7fb-4ee0-90f4-c3957deee01a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:09.518491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea016866-d7fb-4ee0-90f4-c3957deee01a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:09.535737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:09.542769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 55/117] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:09.543913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95487050-887b-4c3f-9c66-41557835f140 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:09.554023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:09.590474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea016866-d7fb-4ee0-90f4-c3957deee01a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:09.592264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea016866-d7fb-4ee0-90f4-c3957deee01a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:09.659696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95487050-887b-4c3f-9c66-41557835f140 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:09.697614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66886223-178a-4135-8437-630678c3af7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:09.698251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66886223-178a-4135-8437-630678c3af7d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:09.698530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66886223-178a-4135-8437-630678c3af7d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:09.698801 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66886223-178a-4135-8437-630678c3af7d None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:09.699963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66886223-178a-4135-8437-630678c3af7d None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:09.702060 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66886223-178a-4135-8437-630678c3af7d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:09.703804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 56/118] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:09 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:09.704616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66886223-178a-4135-8437-630678c3af7d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:09.732922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 63/119] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:09.734026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea016866-d7fb-4ee0-90f4-c3957deee01a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:09.788521 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:09.827301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:09.832050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66886223-178a-4135-8437-630678c3af7d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:09.873122 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37635db3-e78f-4ad2-b2f3-fa86e18d51ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:09.874209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea016866-d7fb-4ee0-90f4-c3957deee01a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:09.874702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37635db3-e78f-4ad2-b2f3-fa86e18d51ec None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:09.875171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37635db3-e78f-4ad2-b2f3-fa86e18d51ec None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:09.875841 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37635db3-e78f-4ad2-b2f3-fa86e18d51ec None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:09.877569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37635db3-e78f-4ad2-b2f3-fa86e18d51ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:09.878047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37635db3-e78f-4ad2-b2f3-fa86e18d51ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:09.879623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37635db3-e78f-4ad2-b2f3-fa86e18d51ec None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:09.898220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 57/120] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:09.898220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37635db3-e78f-4ad2-b2f3-fa86e18d51ec None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:09.915527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c71cf8-9330-4c32-b462-575520503b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:09.916164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c71cf8-9330-4c32-b462-575520503b84 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:09.916440 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c71cf8-9330-4c32-b462-575520503b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:09.916969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c71cf8-9330-4c32-b462-575520503b84 None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:09.917828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c71cf8-9330-4c32-b462-575520503b84 None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:09.919332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c71cf8-9330-4c32-b462-575520503b84 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:09.921581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 64/121] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:09 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:09.922265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c71cf8-9330-4c32-b462-575520503b84 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:10.027220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c71cf8-9330-4c32-b462-575520503b84 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:10.037197 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37635db3-e78f-4ad2-b2f3-fa86e18d51ec None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:10.062898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f8fe3-4895-44bc-b142-961f24ac9655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:10.063566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f8fe3-4895-44bc-b142-961f24ac9655 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:10.063849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f8fe3-4895-44bc-b142-961f24ac9655 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:10.064181 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f8fe3-4895-44bc-b142-961f24ac9655 None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:10.065307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f8fe3-4895-44bc-b142-961f24ac9655 None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:10.066764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f8fe3-4895-44bc-b142-961f24ac9655 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:10.068582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 65/122] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:09 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:10.069278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f8fe3-4895-44bc-b142-961f24ac9655 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:10.072025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:10.076135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:10.087980 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f7e83-8159-41b5-90a4-93fc7e1adcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:10.089247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f7e83-8159-41b5-90a4-93fc7e1adcda None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:10.089742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f7e83-8159-41b5-90a4-93fc7e1adcda None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:10.090317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f7e83-8159-41b5-90a4-93fc7e1adcda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:10.091518 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f7e83-8159-41b5-90a4-93fc7e1adcda None admin] RBAC: Adding query filter params (role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.project.id=96c771455a484eb080ab85c17582f652, user.id=d171ff8c81304d799c869856dff7ee6d) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:10.092088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f7e83-8159-41b5-90a4-93fc7e1adcda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:10.093695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f7e83-8159-41b5-90a4-93fc7e1adcda None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:10.105235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 58/123] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:09 2022] GET /identity/v3/role_assignments?user.id=d171ff8c81304d799c869856dff7ee6d&scope.project.id=96c771455a484eb080ab85c17582f652&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:10.106264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f7e83-8159-41b5-90a4-93fc7e1adcda None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:10.181705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f8fe3-4895-44bc-b142-961f24ac9655 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:10.218151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28237ed7-6763-4b15-9925-b7eb6258706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:10.219011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28237ed7-6763-4b15-9925-b7eb6258706e None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:10.219441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28237ed7-6763-4b15-9925-b7eb6258706e None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:10.219812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28237ed7-6763-4b15-9925-b7eb6258706e None admin] PATH_INFO: `/v3/users/27e08ff6bb314f25a262830c9d8000e2` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:10.221273 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28237ed7-6763-4b15-9925-b7eb6258706e None admin] RBAC: Authorizing `identity:get_user(user_id=27e08ff6bb314f25a262830c9d8000e2)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:10.222904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28237ed7-6763-4b15-9925-b7eb6258706e None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:10.224715 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 66/124] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:10 2022] GET /identity/v3/users/27e08ff6bb314f25a262830c9d8000e2 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:10.225560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28237ed7-6763-4b15-9925-b7eb6258706e None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:10.256575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f7e83-8159-41b5-90a4-93fc7e1adcda None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:10.302628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c27c0-d6ce-46c8-8993-b64b5ba3838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:10.303670 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c27c0-d6ce-46c8-8993-b64b5ba3838e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:10.303955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c27c0-d6ce-46c8-8993-b64b5ba3838e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:10.304232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c27c0-d6ce-46c8-8993-b64b5ba3838e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:10.304973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c27c0-d6ce-46c8-8993-b64b5ba3838e None admin] RBAC: Adding query filter params (role.id=0481df198a1a42f0a00358e0b9a938c5, scope.project.id=96c771455a484eb080ab85c17582f652, user.id=27e08ff6bb314f25a262830c9d8000e2) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:10.305284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c27c0-d6ce-46c8-8993-b64b5ba3838e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:10.308475 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c27c0-d6ce-46c8-8993-b64b5ba3838e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:10.327221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 59/125] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:10 2022] GET /identity/v3/role_assignments?user.id=27e08ff6bb314f25a262830c9d8000e2&scope.project.id=96c771455a484eb080ab85c17582f652&role.id=0481df198a1a42f0a00358e0b9a938c5 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:10.328468 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c27c0-d6ce-46c8-8993-b64b5ba3838e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:10.412928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:10.432486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:10.488361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28237ed7-6763-4b15-9925-b7eb6258706e None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:10.526177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c27c0-d6ce-46c8-8993-b64b5ba3838e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:10.546493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5d033-b49f-46a6-95a2-434f57f71e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:10.546845 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5d033-b49f-46a6-95a2-434f57f71e5f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:10.547112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5d033-b49f-46a6-95a2-434f57f71e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:10.547391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5d033-b49f-46a6-95a2-434f57f71e5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:10.548167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5d033-b49f-46a6-95a2-434f57f71e5f None admin] RBAC: Adding query filter params (role.id=3c56cfa91fba453da353f21d7d6c1189, scope.project.id=96c771455a484eb080ab85c17582f652, user.id=27e08ff6bb314f25a262830c9d8000e2) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:10.549036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5d033-b49f-46a6-95a2-434f57f71e5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:10.549892 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5d033-b49f-46a6-95a2-434f57f71e5f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:10.556628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 67/126] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:10 2022] GET /identity/v3/role_assignments?user.id=27e08ff6bb314f25a262830c9d8000e2&scope.project.id=96c771455a484eb080ab85c17582f652&role.id=3c56cfa91fba453da353f21d7d6c1189 => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:10.557930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841812f-3e61-400c-8721-1cc0efc2626f None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:10.558222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841812f-3e61-400c-8721-1cc0efc2626f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:10.558501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841812f-3e61-400c-8721-1cc0efc2626f None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:10.559783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 68/127] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:10.565778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595f02-d5df-4bde-b57c-c18b6bb545ef None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:10.566066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595f02-d5df-4bde-b57c-c18b6bb545ef None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:10.566349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595f02-d5df-4bde-b57c-c18b6bb545ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:10.569905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcd961-4429-4775-8c3f-513383805fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:10.570939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcd961-4429-4775-8c3f-513383805fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:10.571420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcd961-4429-4775-8c3f-513383805fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:10.571866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcd961-4429-4775-8c3f-513383805fb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:10.590559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-de595f02-d5df-4bde-b57c-c18b6bb545ef None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:10.591808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-de595f02-d5df-4bde-b57c-c18b6bb545ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:10.660725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcd961-4429-4775-8c3f-513383805fb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:10.662235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcd961-4429-4775-8c3f-513383805fb9 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:10.683077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 69/128] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:10.732264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:10.783658 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 60/129] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:10.790247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595f02-d5df-4bde-b57c-c18b6bb545ef None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:10.854799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:10.896159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595f02-d5df-4bde-b57c-c18b6bb545ef None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:10.931048 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067fae2-6620-48ae-9fae-2094e5e10fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:10.931684 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067fae2-6620-48ae-9fae-2094e5e10fdb None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:10.931966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067fae2-6620-48ae-9fae-2094e5e10fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:10.932242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067fae2-6620-48ae-9fae-2094e5e10fdb None admin] PATH_INFO: `/v3/projects/c454bdb8429b4f7ab29e6a38dd69a4b3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:10.933402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067fae2-6620-48ae-9fae-2094e5e10fdb None admin] RBAC: Authorizing `identity:get_project(project_id=c454bdb8429b4f7ab29e6a38dd69a4b3)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:10.934804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067fae2-6620-48ae-9fae-2094e5e10fdb None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:10.937305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 70/130] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:10 2022] GET /identity/v3/projects/c454bdb8429b4f7ab29e6a38dd69a4b3 => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:10.943108 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcd961-4429-4775-8c3f-513383805fb9 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:10.972474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067fae2-6620-48ae-9fae-2094e5e10fdb None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:11.049478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcd961-4429-4775-8c3f-513383805fb9 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:11.077104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067fae2-6620-48ae-9fae-2094e5e10fdb None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:11.083863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a01d4-7efd-4d49-9ef5-6f7a0f590858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:11.084479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a01d4-7efd-4d49-9ef5-6f7a0f590858 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:11.084763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a01d4-7efd-4d49-9ef5-6f7a0f590858 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:11.085037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a01d4-7efd-4d49-9ef5-6f7a0f590858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:11.085757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a01d4-7efd-4d49-9ef5-6f7a0f590858 None admin] RBAC: Adding query filter params (role.id=3c56cfa91fba453da353f21d7d6c1189, scope.project.id=c454bdb8429b4f7ab29e6a38dd69a4b3, user.id=27e08ff6bb314f25a262830c9d8000e2) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:11.086065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a01d4-7efd-4d49-9ef5-6f7a0f590858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:11.087541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a01d4-7efd-4d49-9ef5-6f7a0f590858 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:11.093918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 61/131] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:10 2022] GET /identity/v3/role_assignments?user.id=27e08ff6bb314f25a262830c9d8000e2&scope.project.id=c454bdb8429b4f7ab29e6a38dd69a4b3&role.id=3c56cfa91fba453da353f21d7d6c1189 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:11.109608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:11.111461 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935461dc-87f5-4a9c-8617-6ebb4fb5e18e None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:11.111755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935461dc-87f5-4a9c-8617-6ebb4fb5e18e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:11.112034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935461dc-87f5-4a9c-8617-6ebb4fb5e18e None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:11.113457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 62/132] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:11.116909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b90a-4d32-4ce6-90b9-4f131d2feaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:11.117818 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b90a-4d32-4ce6-90b9-4f131d2feaca None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:11.118269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b90a-4d32-4ce6-90b9-4f131d2feaca None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:11.118740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b90a-4d32-4ce6-90b9-4f131d2feaca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:11.119234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ad849-98d0-4e4f-8407-02abfbe96d9e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:11.119529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ad849-98d0-4e4f-8407-02abfbe96d9e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:11.119831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ad849-98d0-4e4f-8407-02abfbe96d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:11.120640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9b90a-4d32-4ce6-90b9-4f131d2feaca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:11.122328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9b90a-4d32-4ce6-90b9-4f131d2feaca None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:11.124330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 71/133] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:11.130103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b90a-4d32-4ce6-90b9-4f131d2feaca None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:11.140357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-fa9ad849-98d0-4e4f-8407-02abfbe96d9e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:11.141582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-fa9ad849-98d0-4e4f-8407-02abfbe96d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:11.159560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:11.220501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 63/134] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:11.248141 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b90a-4d32-4ce6-90b9-4f131d2feaca None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:11.290378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ed75f-7ab6-4407-a904-245a76c70cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:11.291017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ed75f-7ab6-4407-a904-245a76c70cf1 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:11.291296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ed75f-7ab6-4407-a904-245a76c70cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:11.291571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ed75f-7ab6-4407-a904-245a76c70cf1 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:11.292223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ed75f-7ab6-4407-a904-245a76c70cf1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:11.293752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ed75f-7ab6-4407-a904-245a76c70cf1 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:11.318386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 72/135] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:11 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:11.490874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ad849-98d0-4e4f-8407-02abfbe96d9e None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:11.590216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ad849-98d0-4e4f-8407-02abfbe96d9e None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:11.626133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e69f8b-4c45-419f-bdda-7a57a1b813fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:11.626775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e69f8b-4c45-419f-bdda-7a57a1b813fd None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:11.627141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e69f8b-4c45-419f-bdda-7a57a1b813fd None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:11.627438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e69f8b-4c45-419f-bdda-7a57a1b813fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:11.638048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e69f8b-4c45-419f-bdda-7a57a1b813fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-61e69f8b-4c45-419f-bdda-7a57a1b813fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:11.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:11.644212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:11.644964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 64/136] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:11.650148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ed75f-7ab6-4407-a904-245a76c70cf1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:11.715217 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:11.773254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ed75f-7ab6-4407-a904-245a76c70cf1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:11.808066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf1124-4b9e-475f-ac57-428ea0d70b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:11.808680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf1124-4b9e-475f-ac57-428ea0d70b87 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:11.808976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf1124-4b9e-475f-ac57-428ea0d70b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:11.809223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf1124-4b9e-475f-ac57-428ea0d70b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:11.809881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf1124-4b9e-475f-ac57-428ea0d70b87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:11.810180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf1124-4b9e-475f-ac57-428ea0d70b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:11.811669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf1124-4b9e-475f-ac57-428ea0d70b87 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:11.820324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 73/137] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:11.825853 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e69f8b-4c45-419f-bdda-7a57a1b813fd None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:11.838763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:11.930605 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e69f8b-4c45-419f-bdda-7a57a1b813fd None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:11.965679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f5c59-6a3c-44ba-93df-6e30e993b71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:11.966329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f5c59-6a3c-44ba-93df-6e30e993b71d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:11.966619 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f5c59-6a3c-44ba-93df-6e30e993b71d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:11.966891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f5c59-6a3c-44ba-93df-6e30e993b71d None admin] PATH_INFO: `/v3/users/80c6e88527a243b2970a826e063aeecc` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:11.968593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f5c59-6a3c-44ba-93df-6e30e993b71d None admin] RBAC: Authorizing `identity:get_user(user_id=80c6e88527a243b2970a826e063aeecc)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:11.970076 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f5c59-6a3c-44ba-93df-6e30e993b71d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:11.971717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 65/138] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:11 2022] GET /identity/v3/users/80c6e88527a243b2970a826e063aeecc => generated 289 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:11.977287 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf1124-4b9e-475f-ac57-428ea0d70b87 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:12.083172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf1124-4b9e-475f-ac57-428ea0d70b87 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:12.101651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:12.120405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae793-c960-4716-83b6-4132f70bce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:12.121021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae793-c960-4716-83b6-4132f70bce84 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:12.121299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae793-c960-4716-83b6-4132f70bce84 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:12.121572 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae793-c960-4716-83b6-4132f70bce84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:12.192808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ae793-c960-4716-83b6-4132f70bce84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:12.194343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ae793-c960-4716-83b6-4132f70bce84 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:12.294711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 74/139] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:12.301261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f5c59-6a3c-44ba-93df-6e30e993b71d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:12.388205 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:12.410797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f5c59-6a3c-44ba-93df-6e30e993b71d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:12.445866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d0948-d486-4add-b22d-b3c56ffecb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:12.446510 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d0948-d486-4add-b22d-b3c56ffecb2a None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:12.446791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d0948-d486-4add-b22d-b3c56ffecb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:12.447106 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d0948-d486-4add-b22d-b3c56ffecb2a None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:12.448236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d0948-d486-4add-b22d-b3c56ffecb2a None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:12.449719 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d0948-d486-4add-b22d-b3c56ffecb2a None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:12.451786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 66/140] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:12 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:12.457503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae793-c960-4716-83b6-4132f70bce84 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:12.479710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:12.569437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae793-c960-4716-83b6-4132f70bce84 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:12.606132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20175ad5-9e82-4564-8c96-20a834911c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:12.606736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20175ad5-9e82-4564-8c96-20a834911c64 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:12.607041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20175ad5-9e82-4564-8c96-20a834911c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:12.607315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20175ad5-9e82-4564-8c96-20a834911c64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:12.608657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20175ad5-9e82-4564-8c96-20a834911c64 None admin] RBAC: Adding query filter params (role.id=5981233f19954c69b900f0bbdbb84fd0, scope.project.id=96c771455a484eb080ab85c17582f652, user.id=80c6e88527a243b2970a826e063aeecc) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:12.608962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20175ad5-9e82-4564-8c96-20a834911c64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:12.610404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20175ad5-9e82-4564-8c96-20a834911c64 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:12.616443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 75/141] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:12 2022] GET /identity/v3/role_assignments?user.id=80c6e88527a243b2970a826e063aeecc&scope.project.id=96c771455a484eb080ab85c17582f652&role.id=5981233f19954c69b900f0bbdbb84fd0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:13.626171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67521bb-f279-44db-8078-6a740999b863 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:13.626687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67521bb-f279-44db-8078-6a740999b863 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:13.626761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67521bb-f279-44db-8078-6a740999b863 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:13.628352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 67/142] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:13.633222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f636dd3-4ea7-4fb5-870f-276084053fbb None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:13.633624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f636dd3-4ea7-4fb5-870f-276084053fbb None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:13.633764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f636dd3-4ea7-4fb5-870f-276084053fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:13.653119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-5f636dd3-4ea7-4fb5-870f-276084053fbb None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:13.654355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5f636dd3-4ea7-4fb5-870f-276084053fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:13.729933 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 76/143] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:13.781353 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9b852-af3b-4122-a8ec-931215995e49 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:13.781643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9b852-af3b-4122-a8ec-931215995e49 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:13.781910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9b852-af3b-4122-a8ec-931215995e49 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:13.783426 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 68/144] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:13.788355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d565-6995-42e0-9204-b0ef38d5a4d1 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:13.788663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d565-6995-42e0-9204-b0ef38d5a4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:13.788913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d565-6995-42e0-9204-b0ef38d5a4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:13.806957 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-8684d565-6995-42e0-9204-b0ef38d5a4d1 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:13.808191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8684d565-6995-42e0-9204-b0ef38d5a4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:13.815566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:13.858760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b282f-8412-4c32-838a-3ccd5f72bf5d None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:13.859072 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b282f-8412-4c32-838a-3ccd5f72bf5d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:13.859353 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b282f-8412-4c32-838a-3ccd5f72bf5d None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:13.860692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 69/145] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:13.865799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d095b-5ca8-4eb9-9365-57666b0bec88 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:13.866090 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d095b-5ca8-4eb9-9365-57666b0bec88 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:13.866366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d095b-5ca8-4eb9-9365-57666b0bec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:13.879336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 77/146] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:13.888378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-1e0d095b-5ca8-4eb9-9365-57666b0bec88 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:13.890105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1e0d095b-5ca8-4eb9-9365-57666b0bec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:13.957432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 70/147] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:13.992512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684d565-6995-42e0-9204-b0ef38d5a4d1 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.091408 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684d565-6995-42e0-9204-b0ef38d5a4d1 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.124565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fc257-e131-475c-8551-78b638c76854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.125184 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fc257-e131-475c-8551-78b638c76854 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.125458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fc257-e131-475c-8551-78b638c76854 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.125721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fc257-e131-475c-8551-78b638c76854 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.135671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fc257-e131-475c-8551-78b638c76854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-d70fc257-e131-475c-8551-78b638c76854 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:14.140967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:14.141784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:14.142537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:14.146920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 78/148] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:53:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:14.147521 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d095b-5ca8-4eb9-9365-57666b0bec88 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.147636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fc257-e131-475c-8551-78b638c76854 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.150720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:14.230018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:14.253246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d095b-5ca8-4eb9-9365-57666b0bec88 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.253440 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fc257-e131-475c-8551-78b638c76854 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.286805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba004958-c19d-4e9d-95b7-69ece90057be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.286965 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9d2ba-8467-4a38-942a-81a4681e448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.287461 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba004958-c19d-4e9d-95b7-69ece90057be None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.287557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9d2ba-8467-4a38-942a-81a4681e448a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.287737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba004958-c19d-4e9d-95b7-69ece90057be None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.287826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9d2ba-8467-4a38-942a-81a4681e448a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.288007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba004958-c19d-4e9d-95b7-69ece90057be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.288105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9d2ba-8467-4a38-942a-81a4681e448a None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.288715 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9d2ba-8467-4a38-942a-81a4681e448a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:14.289016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9d2ba-8467-4a38-942a-81a4681e448a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:14.290401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9d2ba-8467-4a38-942a-81a4681e448a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:14.298245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 79/149] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:14.302781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba004958-c19d-4e9d-95b7-69ece90057be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:14.302873 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9d2ba-8467-4a38-942a-81a4681e448a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-ba004958-c19d-4e9d-95b7-69ece90057be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:14.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:14.304246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:14.304999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 71/150] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:53:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:14.305702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba004958-c19d-4e9d-95b7-69ece90057be None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.361369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:14.399132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9d2ba-8467-4a38-942a-81a4681e448a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.404070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba004958-c19d-4e9d-95b7-69ece90057be None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.432547 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c9c38-23e2-44bd-8611-104a7edbf3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.433171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c9c38-23e2-44bd-8611-104a7edbf3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.433450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c9c38-23e2-44bd-8611-104a7edbf3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.433783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c9c38-23e2-44bd-8611-104a7edbf3d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.434821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c9c38-23e2-44bd-8611-104a7edbf3d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:14.436881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c9c38-23e2-44bd-8611-104a7edbf3d1 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:14.437139 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da45522-de05-4047-b48e-7133897a8392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.437759 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da45522-de05-4047-b48e-7133897a8392 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.438036 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da45522-de05-4047-b48e-7133897a8392 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.438305 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da45522-de05-4047-b48e-7133897a8392 None admin] PATH_INFO: `/v3/users/24c299c6c4f6427aa7fedc2d6114df12` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.439015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 80/151] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:14.439422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c9c38-23e2-44bd-8611-104a7edbf3d1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.443561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:14.457467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da45522-de05-4047-b48e-7133897a8392 None admin] RBAC: Authorizing `identity:get_user(user_id=24c299c6c4f6427aa7fedc2d6114df12)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:14.458922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da45522-de05-4047-b48e-7133897a8392 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:14.460608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 72/152] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:14 2022] GET /identity/v3/users/24c299c6c4f6427aa7fedc2d6114df12 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:14.460608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da45522-de05-4047-b48e-7133897a8392 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.542709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c9c38-23e2-44bd-8611-104a7edbf3d1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.558010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da45522-de05-4047-b48e-7133897a8392 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.570278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:14.575574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20055d-b3ba-43aa-b2cd-31cef65c238d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.576175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20055d-b3ba-43aa-b2cd-31cef65c238d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.576981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20055d-b3ba-43aa-b2cd-31cef65c238d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.577250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20055d-b3ba-43aa-b2cd-31cef65c238d None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.577908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20055d-b3ba-43aa-b2cd-31cef65c238d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:14.578209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20055d-b3ba-43aa-b2cd-31cef65c238d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:14.579606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20055d-b3ba-43aa-b2cd-31cef65c238d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:14.587463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 81/153] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:14.588235 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20055d-b3ba-43aa-b2cd-31cef65c238d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.590883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cd52c-3cb0-425e-969f-f5843d2327cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.591532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cd52c-3cb0-425e-969f-f5843d2327cc None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.591817 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cd52c-3cb0-425e-969f-f5843d2327cc None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.592657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cd52c-3cb0-425e-969f-f5843d2327cc None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.593311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cd52c-3cb0-425e-969f-f5843d2327cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:14.594743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cd52c-3cb0-425e-969f-f5843d2327cc None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:14.615729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 73/154] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:14 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:14.615729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cd52c-3cb0-425e-969f-f5843d2327cc None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.674898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:14.709219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20055d-b3ba-43aa-b2cd-31cef65c238d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.741833 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cd52c-3cb0-425e-969f-f5843d2327cc None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.749811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538976d-18ea-43b7-80c0-881597b53377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.750734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538976d-18ea-43b7-80c0-881597b53377 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.751188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538976d-18ea-43b7-80c0-881597b53377 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.751596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538976d-18ea-43b7-80c0-881597b53377 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.753503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538976d-18ea-43b7-80c0-881597b53377 None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:14.755039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538976d-18ea-43b7-80c0-881597b53377 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:14.756955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 82/155] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:14 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:14.762522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538976d-18ea-43b7-80c0-881597b53377 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.784399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80a2b2-42f4-42bd-a6c4-5fc11e782814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.784634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80a2b2-42f4-42bd-a6c4-5fc11e782814 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.784858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80a2b2-42f4-42bd-a6c4-5fc11e782814 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.785119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80a2b2-42f4-42bd-a6c4-5fc11e782814 None admin] PATH_INFO: `/v3/users/24c299c6c4f6427aa7fedc2d6114df12` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.786366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80a2b2-42f4-42bd-a6c4-5fc11e782814 None admin] RBAC: Authorizing `identity:get_user(user_id=24c299c6c4f6427aa7fedc2d6114df12)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:14.787835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80a2b2-42f4-42bd-a6c4-5fc11e782814 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:14.789663 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 74/156] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:14 2022] GET /identity/v3/users/24c299c6c4f6427aa7fedc2d6114df12 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:14.795362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80a2b2-42f4-42bd-a6c4-5fc11e782814 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:14.833954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:14.877158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538976d-18ea-43b7-80c0-881597b53377 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.921328 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80a2b2-42f4-42bd-a6c4-5fc11e782814 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:14.924839 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d4c6c-b5ae-4b0f-8437-b4170821cf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.925557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d4c6c-b5ae-4b0f-8437-b4170821cf22 None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.925781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d4c6c-b5ae-4b0f-8437-b4170821cf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.926070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d4c6c-b5ae-4b0f-8437-b4170821cf22 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/users/24c299c6c4f6427aa7fedc2d6114df12/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.928641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d4c6c-b5ae-4b0f-8437-b4170821cf22 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ccdf5ff122f47538ebe72016c0cab36, user_id=24c299c6c4f6427aa7fedc2d6114df12, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:14.930106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d4c6c-b5ae-4b0f-8437-b4170821cf22 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:14.943964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:14.946559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 83/157] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:14 2022] PUT /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/users/24c299c6c4f6427aa7fedc2d6114df12/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 18:53:14.968523 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182db19d-e643-4511-b3f4-99ecb75c8dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:14.969261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182db19d-e643-4511-b3f4-99ecb75c8dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:14.969543 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182db19d-e643-4511-b3f4-99ecb75c8dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:14.969814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182db19d-e643-4511-b3f4-99ecb75c8dc7 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:14.971039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182db19d-e643-4511-b3f4-99ecb75c8dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:14.972557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182db19d-e643-4511-b3f4-99ecb75c8dc7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:14.974562 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 75/158] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:14 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:14.980478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d4c6c-b5ae-4b0f-8437-b4170821cf22 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:15.102250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d4c6c-b5ae-4b0f-8437-b4170821cf22 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:15.127755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:15.140603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8d4e2-acc2-4609-8288-ed378cecb32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:15.141263 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d4e2-acc2-4609-8288-ed378cecb32b None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:15.141543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d4e2-acc2-4609-8288-ed378cecb32b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:15.141809 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d4e2-acc2-4609-8288-ed378cecb32b None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/users/24c299c6c4f6427aa7fedc2d6114df12/roles/0481df198a1a42f0a00358e0b9a938c5` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:15.145414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8d4e2-acc2-4609-8288-ed378cecb32b None admin] RBAC: Authorizing `identity:create_grant(project_id=2ccdf5ff122f47538ebe72016c0cab36, user_id=24c299c6c4f6427aa7fedc2d6114df12, role_id=0481df198a1a42f0a00358e0b9a938c5)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:15.146904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8d4e2-acc2-4609-8288-ed378cecb32b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:15.164442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 84/159] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:14 2022] PUT /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/users/24c299c6c4f6427aa7fedc2d6114df12/roles/0481df198a1a42f0a00358e0b9a938c5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:16.898620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77328377-e4c1-4a78-887b-a5f71b600d4d None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:16.899221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77328377-e4c1-4a78-887b-a5f71b600d4d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:16.899221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77328377-e4c1-4a78-887b-a5f71b600d4d None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:16.900645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 76/160] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:16.907135 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067a3f3-c788-4974-8a6c-f25dab5b63e0 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:16.908249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067a3f3-c788-4974-8a6c-f25dab5b63e0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:16.908735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067a3f3-c788-4974-8a6c-f25dab5b63e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:16.928379 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-e067a3f3-c788-4974-8a6c-f25dab5b63e0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:16.929952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e067a3f3-c788-4974-8a6c-f25dab5b63e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:16.998824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 85/161] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:17.033893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c82486-0b15-4d51-a807-4f122389b566 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.034252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c82486-0b15-4d51-a807-4f122389b566 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.034429 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c82486-0b15-4d51-a807-4f122389b566 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.036071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 77/162] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:17.042021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994b01-2367-4e6e-be30-5445c19ef7ff None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.042348 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994b01-2367-4e6e-be30-5445c19ef7ff None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.042579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994b01-2367-4e6e-be30-5445c19ef7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.062651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b2994b01-2367-4e6e-be30-5445c19ef7ff None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:17.063908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b2994b01-2367-4e6e-be30-5445c19ef7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:17.135974 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 86/163] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:17.164339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726150e8-1b8a-4708-b411-843d8f99ec0a None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.164644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726150e8-1b8a-4708-b411-843d8f99ec0a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.164886 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726150e8-1b8a-4708-b411-843d8f99ec0a None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.166359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 78/164] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:17.171528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466908-f030-4e8b-8cfd-520aedd6e639 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.171816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466908-f030-4e8b-8cfd-520aedd6e639 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.172089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466908-f030-4e8b-8cfd-520aedd6e639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.190621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b6466908-f030-4e8b-8cfd-520aedd6e639 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:17.191825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b6466908-f030-4e8b-8cfd-520aedd6e639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:17.230931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:17.264370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 87/165] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:17.283293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726150e8-1b8a-4708-b411-843d8f99ec0a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:17.400930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726150e8-1b8a-4708-b411-843d8f99ec0a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:17.433427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6466908-f030-4e8b-8cfd-520aedd6e639 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:17.439988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd1116-473d-47d1-a44a-d584b16c65ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:17.440624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd1116-473d-47d1-a44a-d584b16c65ab None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.440902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd1116-473d-47d1-a44a-d584b16c65ab None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.441170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd1116-473d-47d1-a44a-d584b16c65ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.453190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd1116-473d-47d1-a44a-d584b16c65ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-a8fd1116-473d-47d1-a44a-d584b16c65ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:17.464531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:17.465439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:17.466253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 79/166] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:17.467070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3b1cd-1f98-4e16-8555-c9496f050745 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.467070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3b1cd-1f98-4e16-8555-c9496f050745 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.467606 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3b1cd-1f98-4e16-8555-c9496f050745 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.468800 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 80/167] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:17.473407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3b1cd-1f98-4e16-8555-c9496f050745 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:17.523057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:17.541029 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6466908-f030-4e8b-8cfd-520aedd6e639 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:17.574864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3c662-6a06-471a-8407-3fe73da32d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:17.575514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3c662-6a06-471a-8407-3fe73da32d5d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.575796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3c662-6a06-471a-8407-3fe73da32d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.576063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3c662-6a06-471a-8407-3fe73da32d5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.577187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3c662-6a06-471a-8407-3fe73da32d5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:17.578624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3c662-6a06-471a-8407-3fe73da32d5d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:17.579982 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3b1cd-1f98-4e16-8555-c9496f050745 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:17.580211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 88/168] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:17.584879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f43dc-a980-4b05-91e3-14813293241e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.584879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f43dc-a980-4b05-91e3-14813293241e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.584879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f43dc-a980-4b05-91e3-14813293241e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.600336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-334f43dc-a980-4b05-91e3-14813293241e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:17.601526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-334f43dc-a980-4b05-91e3-14813293241e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:17.613893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b2c8f-53b8-4a3c-a661-d5399ccd0ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:17.614500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b2c8f-53b8-4a3c-a661-d5399ccd0ece None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.614775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b2c8f-53b8-4a3c-a661-d5399ccd0ece None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.615072 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b2c8f-53b8-4a3c-a661-d5399ccd0ece None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.615733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b2c8f-53b8-4a3c-a661-d5399ccd0ece None admin] RBAC: Adding query filter params (name=member) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:17.616034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b2c8f-53b8-4a3c-a661-d5399ccd0ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:17.617405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b2c8f-53b8-4a3c-a661-d5399ccd0ece None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:17.625712 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 81/169] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:17.625712 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b2c8f-53b8-4a3c-a661-d5399ccd0ece None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:17.635630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:17.668233 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 89/170] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:17.669277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f43dc-a980-4b05-91e3-14813293241e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:17.728176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b2c8f-53b8-4a3c-a661-d5399ccd0ece None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:17.739231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:17.768940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92162b-9e7c-44db-b0a5-73fc4d0826ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:17.769662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92162b-9e7c-44db-b0a5-73fc4d0826ea None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.769942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92162b-9e7c-44db-b0a5-73fc4d0826ea None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.770258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92162b-9e7c-44db-b0a5-73fc4d0826ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.778299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f43dc-a980-4b05-91e3-14813293241e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:17.780632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92162b-9e7c-44db-b0a5-73fc4d0826ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-fe92162b-9e7c-44db-b0a5-73fc4d0826ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:17.785900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:17.786775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:17.788641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 82/171] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:53:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:17.792267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92162b-9e7c-44db-b0a5-73fc4d0826ea None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:17.815355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de0c72-4650-4b3b-88a9-0c780cbc0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:17.815958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de0c72-4650-4b3b-88a9-0c780cbc0979 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.816231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de0c72-4650-4b3b-88a9-0c780cbc0979 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.816498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de0c72-4650-4b3b-88a9-0c780cbc0979 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.817130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de0c72-4650-4b3b-88a9-0c780cbc0979 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:17.818554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de0c72-4650-4b3b-88a9-0c780cbc0979 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:17.840546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 90/172] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:17 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:17.841168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de0c72-4650-4b3b-88a9-0c780cbc0979 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:17.883025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:17.916888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92162b-9e7c-44db-b0a5-73fc4d0826ea None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:17.961267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f43c4-6a64-4595-b603-f86f16fa4f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:17.961832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f43c4-6a64-4595-b603-f86f16fa4f47 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:17.962109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f43c4-6a64-4595-b603-f86f16fa4f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:17.962377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f43c4-6a64-4595-b603-f86f16fa4f47 None admin] PATH_INFO: `/v3/users/6c1477d2dd0f422bb01445d2ac98c7f3` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:17.978490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f43c4-6a64-4595-b603-f86f16fa4f47 None admin] RBAC: Authorizing `identity:get_user(user_id=6c1477d2dd0f422bb01445d2ac98c7f3)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:17.980051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f43c4-6a64-4595-b603-f86f16fa4f47 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:17.981383 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de0c72-4650-4b3b-88a9-0c780cbc0979 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:17.987357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 83/173] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:17 2022] GET /identity/v3/users/6c1477d2dd0f422bb01445d2ac98c7f3 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:17.987357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f43c4-6a64-4595-b603-f86f16fa4f47 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:18.016703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:18.026342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198a883-2b94-4e81-83ad-3458b2be18a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:18.026708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198a883-2b94-4e81-83ad-3458b2be18a7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:18.026998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198a883-2b94-4e81-83ad-3458b2be18a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:18.027345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198a883-2b94-4e81-83ad-3458b2be18a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:18.028108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198a883-2b94-4e81-83ad-3458b2be18a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:18.028317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198a883-2b94-4e81-83ad-3458b2be18a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:18.030236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198a883-2b94-4e81-83ad-3458b2be18a7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:18.039501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 91/174] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:18.040459 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198a883-2b94-4e81-83ad-3458b2be18a7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:18.097361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f43c4-6a64-4595-b603-f86f16fa4f47 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:18.153145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cf7e5-96dd-407e-b4c8-b4190e3f441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:18.153791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cf7e5-96dd-407e-b4c8-b4190e3f441b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:18.154070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cf7e5-96dd-407e-b4c8-b4190e3f441b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:18.154392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cf7e5-96dd-407e-b4c8-b4190e3f441b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:18.160427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:18.169380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cf7e5-96dd-407e-b4c8-b4190e3f441b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:18.174067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198a883-2b94-4e81-83ad-3458b2be18a7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-ab7cf7e5-96dd-407e-b4c8-b4190e3f441b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:18.175416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:18.176447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 84/175] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:53:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:18.177214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cf7e5-96dd-407e-b4c8-b4190e3f441b None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:18.223990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf127c4-0cdb-469a-8d1a-13c3eb7501fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:18.224914 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf127c4-0cdb-469a-8d1a-13c3eb7501fa None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:18.225349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf127c4-0cdb-469a-8d1a-13c3eb7501fa None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:18.225783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf127c4-0cdb-469a-8d1a-13c3eb7501fa None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:18.227202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf127c4-0cdb-469a-8d1a-13c3eb7501fa None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:18.229341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf127c4-0cdb-469a-8d1a-13c3eb7501fa None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:18.231653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 92/176] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:18 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:18.233521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf127c4-0cdb-469a-8d1a-13c3eb7501fa None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:18.284819 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:18.302301 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cf7e5-96dd-407e-b4c8-b4190e3f441b None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:18.343437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae22e4-85ab-48e8-9719-d58537ad33fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:18.344232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae22e4-85ab-48e8-9719-d58537ad33fb None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:18.344600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae22e4-85ab-48e8-9719-d58537ad33fb None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:18.344963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae22e4-85ab-48e8-9719-d58537ad33fb None admin] PATH_INFO: `/v3/users/24c299c6c4f6427aa7fedc2d6114df12` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:18.346213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae22e4-85ab-48e8-9719-d58537ad33fb None admin] RBAC: Authorizing `identity:get_user(user_id=24c299c6c4f6427aa7fedc2d6114df12)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:18.347808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae22e4-85ab-48e8-9719-d58537ad33fb None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:18.349497 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 85/177] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:18 2022] GET /identity/v3/users/24c299c6c4f6427aa7fedc2d6114df12 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:18.350411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae22e4-85ab-48e8-9719-d58537ad33fb None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:18.382962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf127c4-0cdb-469a-8d1a-13c3eb7501fa None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:18.427443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff113cc4-14de-45d0-a03c-7b0157fdaad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:18.428226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff113cc4-14de-45d0-a03c-7b0157fdaad2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:18.428502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff113cc4-14de-45d0-a03c-7b0157fdaad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:18.428779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff113cc4-14de-45d0-a03c-7b0157fdaad2 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:18.429439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff113cc4-14de-45d0-a03c-7b0157fdaad2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:18.429740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff113cc4-14de-45d0-a03c-7b0157fdaad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:18.431172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff113cc4-14de-45d0-a03c-7b0157fdaad2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:18.441753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 93/178] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:18.447150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff113cc4-14de-45d0-a03c-7b0157fdaad2 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:18.465156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae22e4-85ab-48e8-9719-d58537ad33fb None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:18.467146 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:18.512582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53dd60-8750-46d5-ac68-1ec94ecaf831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:18.513527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53dd60-8750-46d5-ac68-1ec94ecaf831 None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:18.513845 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53dd60-8750-46d5-ac68-1ec94ecaf831 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:18.514123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53dd60-8750-46d5-ac68-1ec94ecaf831 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/users/6c1477d2dd0f422bb01445d2ac98c7f3/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:18.516877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53dd60-8750-46d5-ac68-1ec94ecaf831 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ccdf5ff122f47538ebe72016c0cab36, user_id=6c1477d2dd0f422bb01445d2ac98c7f3, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:18.518792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53dd60-8750-46d5-ac68-1ec94ecaf831 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:18.538063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 86/179] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:18 2022] PUT /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/users/6c1477d2dd0f422bb01445d2ac98c7f3/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:18.538885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53dd60-8750-46d5-ac68-1ec94ecaf831 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:18.575395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:18.597269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff113cc4-14de-45d0-a03c-7b0157fdaad2 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:18.690420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53dd60-8750-46d5-ac68-1ec94ecaf831 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:18.731349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f6cad-09f8-4ef6-993d-6fca11f6fb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:18.731984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f6cad-09f8-4ef6-993d-6fca11f6fb58 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:18.732263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f6cad-09f8-4ef6-993d-6fca11f6fb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:18.733190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f6cad-09f8-4ef6-993d-6fca11f6fb58 None admin] PATH_INFO: `/v3/users/24c299c6c4f6427aa7fedc2d6114df12` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:18.734364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f6cad-09f8-4ef6-993d-6fca11f6fb58 None admin] RBAC: Authorizing `identity:get_user(user_id=24c299c6c4f6427aa7fedc2d6114df12)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:18.735975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f6cad-09f8-4ef6-993d-6fca11f6fb58 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:18.737691 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 87/180] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:18 2022] GET /identity/v3/users/24c299c6c4f6427aa7fedc2d6114df12 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:18.739479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3b46c-de20-4961-b608-8da2b39e1aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:18.740168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3b46c-de20-4961-b608-8da2b39e1aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:18.740450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3b46c-de20-4961-b608-8da2b39e1aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:18.740721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3b46c-de20-4961-b608-8da2b39e1aa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:18.744284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f6cad-09f8-4ef6-993d-6fca11f6fb58 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:18.844145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:18.847403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3b46c-de20-4961-b608-8da2b39e1aa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:18.849153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3b46c-de20-4961-b608-8da2b39e1aa9 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:18.867828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f6cad-09f8-4ef6-993d-6fca11f6fb58 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:18.874505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:18.904251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30351c2e-a3a3-46cc-82fa-cced7bed4b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:18.905550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351c2e-a3a3-46cc-82fa-cced7bed4b85 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:18.905936 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351c2e-a3a3-46cc-82fa-cced7bed4b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:18.906238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351c2e-a3a3-46cc-82fa-cced7bed4b85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:18.974666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30351c2e-a3a3-46cc-82fa-cced7bed4b85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:18.983991 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30351c2e-a3a3-46cc-82fa-cced7bed4b85 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:19.043730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 94/181] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:19.049272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3b46c-de20-4961-b608-8da2b39e1aa9 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:19.072812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:19.093419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 88/182] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:19.101765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30351c2e-a3a3-46cc-82fa-cced7bed4b85 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:19.289749 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30351c2e-a3a3-46cc-82fa-cced7bed4b85 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:19.292331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3b46c-de20-4961-b608-8da2b39e1aa9 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:19.333851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:19.358535 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0507972-5a28-408a-a707-8f1c1122cadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:19.359215 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0507972-5a28-408a-a707-8f1c1122cadd None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:19.359418 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0507972-5a28-408a-a707-8f1c1122cadd None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:19.359665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0507972-5a28-408a-a707-8f1c1122cadd None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:19.362267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0507972-5a28-408a-a707-8f1c1122cadd None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:19.363982 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0507972-5a28-408a-a707-8f1c1122cadd None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:19.373793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0606c02-cac4-4757-8365-069c86972536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:19.374751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606c02-cac4-4757-8365-069c86972536 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:19.375254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606c02-cac4-4757-8365-069c86972536 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:19.375740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606c02-cac4-4757-8365-069c86972536 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:19.377270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0606c02-cac4-4757-8365-069c86972536 None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:19.378913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0606c02-cac4-4757-8365-069c86972536 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:19.379172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 89/183] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:19 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:19.381052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 95/184] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:19 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:19.383255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0507972-5a28-408a-a707-8f1c1122cadd None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:19.387101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0606c02-cac4-4757-8365-069c86972536 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:19.476349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:19.508723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0606c02-cac4-4757-8365-069c86972536 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:19.548015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21da394-c4a9-4043-8546-11a3fe26bcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:19.548980 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da394-c4a9-4043-8546-11a3fe26bcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:19.549438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da394-c4a9-4043-8546-11a3fe26bcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:19.549905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da394-c4a9-4043-8546-11a3fe26bcd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:19.550806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21da394-c4a9-4043-8546-11a3fe26bcd2 None admin] RBAC: Adding query filter params (role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.project.id=2ccdf5ff122f47538ebe72016c0cab36, user.id=24c299c6c4f6427aa7fedc2d6114df12) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:19.551325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21da394-c4a9-4043-8546-11a3fe26bcd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:19.552948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21da394-c4a9-4043-8546-11a3fe26bcd2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:19.559913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 96/185] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:19 2022] GET /identity/v3/role_assignments?user.id=24c299c6c4f6427aa7fedc2d6114df12&scope.project.id=2ccdf5ff122f47538ebe72016c0cab36&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:19.588772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0507972-5a28-408a-a707-8f1c1122cadd None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:19.641136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9a43b-8b32-4a37-9ad1-a23a03f03e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:19.641671 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9a43b-8b32-4a37-9ad1-a23a03f03e50 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:19.641947 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9a43b-8b32-4a37-9ad1-a23a03f03e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:19.642219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9a43b-8b32-4a37-9ad1-a23a03f03e50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:19.642949 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9a43b-8b32-4a37-9ad1-a23a03f03e50 None admin] RBAC: Adding query filter params (role.id=0481df198a1a42f0a00358e0b9a938c5, scope.project.id=2ccdf5ff122f47538ebe72016c0cab36, user.id=24c299c6c4f6427aa7fedc2d6114df12) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:19.643534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9a43b-8b32-4a37-9ad1-a23a03f03e50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:19.644980 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9a43b-8b32-4a37-9ad1-a23a03f03e50 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:19.651349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 90/186] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:19 2022] GET /identity/v3/role_assignments?user.id=24c299c6c4f6427aa7fedc2d6114df12&scope.project.id=2ccdf5ff122f47538ebe72016c0cab36&role.id=0481df198a1a42f0a00358e0b9a938c5 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:20.326670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984bf7-df3f-4b0a-847d-6e52118f60a9 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:20.327250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984bf7-df3f-4b0a-847d-6e52118f60a9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:20.327250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984bf7-df3f-4b0a-847d-6e52118f60a9 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:20.328724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 97/187] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:20.333709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba11638-f690-4367-9b2a-7cecab9472cd None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:20.334094 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba11638-f690-4367-9b2a-7cecab9472cd None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:20.334249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba11638-f690-4367-9b2a-7cecab9472cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:20.337042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b006c44-c27b-46b3-80be-fbceb1a368a7 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:20.337389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b006c44-c27b-46b3-80be-fbceb1a368a7 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:20.337591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b006c44-c27b-46b3-80be-fbceb1a368a7 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:20.339012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 98/188] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:20.344250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd08f2-525e-4d1c-9ad8-6011f6b6e8c0 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:20.344528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd08f2-525e-4d1c-9ad8-6011f6b6e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:20.344796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd08f2-525e-4d1c-9ad8-6011f6b6e8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:20.353263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-5ba11638-f690-4367-9b2a-7cecab9472cd None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:20.354445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5ba11638-f690-4367-9b2a-7cecab9472cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:20.363800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-65fd08f2-525e-4d1c-9ad8-6011f6b6e8c0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:20.364945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-65fd08f2-525e-4d1c-9ad8-6011f6b6e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:20.413024 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:20.429019 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 91/189] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:20.443016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 99/190] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:20.694537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba11638-f690-4367-9b2a-7cecab9472cd None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:20.703113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd08f2-525e-4d1c-9ad8-6011f6b6e8c0 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:20.753254 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:20.806418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd08f2-525e-4d1c-9ad8-6011f6b6e8c0 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:20.806756 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba11638-f690-4367-9b2a-7cecab9472cd None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:20.842695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df8715-e0a5-4e5b-b54c-6deb7b986e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:20.843356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df8715-e0a5-4e5b-b54c-6deb7b986e43 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:20.843637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df8715-e0a5-4e5b-b54c-6deb7b986e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:20.843744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134f64a-c4d5-41e0-94e1-9614348339f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:20.843926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df8715-e0a5-4e5b-b54c-6deb7b986e43 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:20.844319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134f64a-c4d5-41e0-94e1-9614348339f5 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:20.844597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134f64a-c4d5-41e0-94e1-9614348339f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:20.844866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134f64a-c4d5-41e0-94e1-9614348339f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:20.845957 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134f64a-c4d5-41e0-94e1-9614348339f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:20.847432 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134f64a-c4d5-41e0-94e1-9614348339f5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:20.848993 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 100/191] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:20.854640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134f64a-c4d5-41e0-94e1-9614348339f5 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:20.856430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df8715-e0a5-4e5b-b54c-6deb7b986e43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-24df8715-e0a5-4e5b-b54c-6deb7b986e43 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:20.862037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:20.862897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:20.863835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 92/192] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:20.868739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df8715-e0a5-4e5b-b54c-6deb7b986e43 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:20.910970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:20.966421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134f64a-c4d5-41e0-94e1-9614348339f5 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:20.977601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df8715-e0a5-4e5b-b54c-6deb7b986e43 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:20.979889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:21.007153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0a389-87cc-49f8-beef-fdca9f4bf0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:21.007764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0a389-87cc-49f8-beef-fdca9f4bf0f0 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.008042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0a389-87cc-49f8-beef-fdca9f4bf0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.008322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0a389-87cc-49f8-beef-fdca9f4bf0f0 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.008970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0a389-87cc-49f8-beef-fdca9f4bf0f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:21.010415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0a389-87cc-49f8-beef-fdca9f4bf0f0 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:21.013741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39ddc-fb03-4d5f-a96d-08216f7a376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:21.014341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39ddc-fb03-4d5f-a96d-08216f7a376c None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.014616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39ddc-fb03-4d5f-a96d-08216f7a376c None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.014885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39ddc-fb03-4d5f-a96d-08216f7a376c None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.015573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e39ddc-fb03-4d5f-a96d-08216f7a376c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:21.015870 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e39ddc-fb03-4d5f-a96d-08216f7a376c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:21.017269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e39ddc-fb03-4d5f-a96d-08216f7a376c None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:21.026455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 93/193] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:21.027996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7637fd0-605a-42f4-b0f7-366201de2c99 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.028280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7637fd0-605a-42f4-b0f7-366201de2c99 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.028555 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7637fd0-605a-42f4-b0f7-366201de2c99 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.030555 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 94/194] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:21.032202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7637fd0-605a-42f4-b0f7-366201de2c99 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:21.033859 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 101/195] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:20 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:21.035572 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e39b53-03d0-47f2-b95b-389f0a2ee242 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.035927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e39b53-03d0-47f2-b95b-389f0a2ee242 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.036524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e39b53-03d0-47f2-b95b-389f0a2ee242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.058662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b2e39b53-03d0-47f2-b95b-389f0a2ee242 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:21.059916 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b2e39b53-03d0-47f2-b95b-389f0a2ee242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:21.135161 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 102/196] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:21.157781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7637fd0-605a-42f4-b0f7-366201de2c99 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:21.195660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b6d3-d628-4124-862f-df96fc88f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:21.196304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b6d3-d628-4124-862f-df96fc88f81f None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.196594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b6d3-d628-4124-862f-df96fc88f81f None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.196848 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b6d3-d628-4124-862f-df96fc88f81f None admin] PATH_INFO: `/v3/users/fe029248c81c46329b912053bf2894bf` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.212045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92b6d3-d628-4124-862f-df96fc88f81f None admin] RBAC: Authorizing `identity:get_user(user_id=fe029248c81c46329b912053bf2894bf)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:21.213175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92b6d3-d628-4124-862f-df96fc88f81f None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:21.214432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 95/197] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:21 2022] GET /identity/v3/users/fe029248c81c46329b912053bf2894bf => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:21.220245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e39b53-03d0-47f2-b95b-389f0a2ee242 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:21.255724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:21.324500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e39b53-03d0-47f2-b95b-389f0a2ee242 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:21.360753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3fcb7-eedb-4c15-97f3-0818ac7cf2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:21.361390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3fcb7-eedb-4c15-97f3-0818ac7cf2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.361651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3fcb7-eedb-4c15-97f3-0818ac7cf2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.361920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3fcb7-eedb-4c15-97f3-0818ac7cf2a7 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.363049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3fcb7-eedb-4c15-97f3-0818ac7cf2a7 None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:21.364496 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3fcb7-eedb-4c15-97f3-0818ac7cf2a7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:21.366130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 103/198] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:21 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:21.371925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b6d3-d628-4124-862f-df96fc88f81f None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:21.401245 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:21.413690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3fcb7-eedb-4c15-97f3-0818ac7cf2a7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:21.495899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b6d3-d628-4124-862f-df96fc88f81f None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:21.534653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444093db-19e1-42f7-a1f5-b41b10286173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:21.535578 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444093db-19e1-42f7-a1f5-b41b10286173 None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.536122 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444093db-19e1-42f7-a1f5-b41b10286173 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.536643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444093db-19e1-42f7-a1f5-b41b10286173 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/users/fe029248c81c46329b912053bf2894bf/roles/5981233f19954c69b900f0bbdbb84fd0` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.538854 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444093db-19e1-42f7-a1f5-b41b10286173 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ccdf5ff122f47538ebe72016c0cab36, user_id=fe029248c81c46329b912053bf2894bf, role_id=5981233f19954c69b900f0bbdbb84fd0)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:21.540719 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444093db-19e1-42f7-a1f5-b41b10286173 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:21.543395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3fcb7-eedb-4c15-97f3-0818ac7cf2a7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:21.546301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:21.556799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 96/199] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:21 2022] PUT /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/users/fe029248c81c46329b912053bf2894bf/roles/5981233f19954c69b900f0bbdbb84fd0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:21.585626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912a00c-3cba-4e6f-a709-f5ee26395d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:21.586281 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912a00c-3cba-4e6f-a709-f5ee26395d5d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.586559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912a00c-3cba-4e6f-a709-f5ee26395d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.586828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912a00c-3cba-4e6f-a709-f5ee26395d5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.599851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912a00c-3cba-4e6f-a709-f5ee26395d5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-1912a00c-3cba-4e6f-a709-f5ee26395d5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:21.605060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:21.606016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:21.606778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 104/200] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:21.612813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444093db-19e1-42f7-a1f5-b41b10286173 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:21.713237 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:21.733200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444093db-19e1-42f7-a1f5-b41b10286173 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:21.772002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccd034-19a8-4981-8779-9dcc101ee4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:21.773010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccd034-19a8-4981-8779-9dcc101ee4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.773290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccd034-19a8-4981-8779-9dcc101ee4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.773560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccd034-19a8-4981-8779-9dcc101ee4f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.774231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccd034-19a8-4981-8779-9dcc101ee4f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:21.774530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccd034-19a8-4981-8779-9dcc101ee4f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:21.775940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccd034-19a8-4981-8779-9dcc101ee4f3 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:21.785935 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 97/201] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:21.791489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912a00c-3cba-4e6f-a709-f5ee26395d5d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:21.896384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912a00c-3cba-4e6f-a709-f5ee26395d5d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:21.932190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d1557-0cc3-40a3-8279-ddaf771cfc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:21.932829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d1557-0cc3-40a3-8279-ddaf771cfc14 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:21.933106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d1557-0cc3-40a3-8279-ddaf771cfc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:21.933380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d1557-0cc3-40a3-8279-ddaf771cfc14 None admin] PATH_INFO: `/v3/users/6c1477d2dd0f422bb01445d2ac98c7f3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:21.934519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d1557-0cc3-40a3-8279-ddaf771cfc14 None admin] RBAC: Authorizing `identity:get_user(user_id=6c1477d2dd0f422bb01445d2ac98c7f3)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:21.936052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d1557-0cc3-40a3-8279-ddaf771cfc14 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:21.937643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 105/202] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:21 2022] GET /identity/v3/users/6c1477d2dd0f422bb01445d2ac98c7f3 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:21.943295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccd034-19a8-4981-8779-9dcc101ee4f3 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:22.047411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccd034-19a8-4981-8779-9dcc101ee4f3 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:22.082336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49671090-70e2-420a-95d1-f311cf4d6641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:22.082953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49671090-70e2-420a-95d1-f311cf4d6641 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:22.083317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49671090-70e2-420a-95d1-f311cf4d6641 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:22.083591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49671090-70e2-420a-95d1-f311cf4d6641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:22.100873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:22.158551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49671090-70e2-420a-95d1-f311cf4d6641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:22.160102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49671090-70e2-420a-95d1-f311cf4d6641 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:22.260070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 98/203] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 28 18:53:22.266230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d1557-0cc3-40a3-8279-ddaf771cfc14 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:22.312001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:22.371373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d1557-0cc3-40a3-8279-ddaf771cfc14 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:22.408998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f635cf-4979-4ecf-83b5-c40604c029e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:22.409627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f635cf-4979-4ecf-83b5-c40604c029e4 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:22.409903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f635cf-4979-4ecf-83b5-c40604c029e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:22.410175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f635cf-4979-4ecf-83b5-c40604c029e4 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:22.411327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f635cf-4979-4ecf-83b5-c40604c029e4 None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:22.412779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f635cf-4979-4ecf-83b5-c40604c029e4 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:22.414402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 106/204] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:22 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:22.420384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49671090-70e2-420a-95d1-f311cf4d6641 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:22.469049 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:22.525666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49671090-70e2-420a-95d1-f311cf4d6641 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:22.563063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5a6b4-59e4-4841-a419-5df631a05687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:22.563594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5a6b4-59e4-4841-a419-5df631a05687 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:22.563879 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5a6b4-59e4-4841-a419-5df631a05687 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:22.564149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5a6b4-59e4-4841-a419-5df631a05687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:22.564871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5a6b4-59e4-4841-a419-5df631a05687 None admin] RBAC: Adding query filter params (role.id=3c56cfa91fba453da353f21d7d6c1189, scope.project.id=2ccdf5ff122f47538ebe72016c0cab36, user.id=6c1477d2dd0f422bb01445d2ac98c7f3) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:22.565175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5a6b4-59e4-4841-a419-5df631a05687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:22.566881 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5a6b4-59e4-4841-a419-5df631a05687 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:22.573085 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 99/205] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:22 2022] GET /identity/v3/role_assignments?user.id=6c1477d2dd0f422bb01445d2ac98c7f3&scope.project.id=2ccdf5ff122f47538ebe72016c0cab36&role.id=3c56cfa91fba453da353f21d7d6c1189 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:23.400135 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c77853-a36e-4a8b-8f62-db10405378f8 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:23.400657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c77853-a36e-4a8b-8f62-db10405378f8 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:23.400657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c77853-a36e-4a8b-8f62-db10405378f8 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:23.402175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 107/206] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:23.408028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ce71f-2c73-4f4f-a51f-2371f25f8727 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:23.408310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ce71f-2c73-4f4f-a51f-2371f25f8727 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:23.408576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ce71f-2c73-4f4f-a51f-2371f25f8727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:23.426430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-809ce71f-2c73-4f4f-a51f-2371f25f8727 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:23.427626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-809ce71f-2c73-4f4f-a51f-2371f25f8727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:23.441602 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54fe2e-e0e1-4e84-a92b-a6bc700aa66b None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:23.441982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54fe2e-e0e1-4e84-a92b-a6bc700aa66b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:23.442158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54fe2e-e0e1-4e84-a92b-a6bc700aa66b None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:23.443663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 108/207] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:23.448500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b29d19-b367-4cff-9920-5824f90541f5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:23.448782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b29d19-b367-4cff-9920-5824f90541f5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:23.449056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b29d19-b367-4cff-9920-5824f90541f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:23.467726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-55b29d19-b367-4cff-9920-5824f90541f5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:23.468909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-55b29d19-b367-4cff-9920-5824f90541f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:23.507093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 100/208] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:23.540878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 109/209] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:23.772904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ce71f-2c73-4f4f-a51f-2371f25f8727 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:23.806210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b29d19-b367-4cff-9920-5824f90541f5 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:23.814760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:23.836664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:23.876536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ce71f-2c73-4f4f-a51f-2371f25f8727 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:23.908494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b29d19-b367-4cff-9920-5824f90541f5 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:23.910210 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26701f3b-3cb5-41e5-8210-57ed2bd68746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:23.910838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26701f3b-3cb5-41e5-8210-57ed2bd68746 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:23.911138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26701f3b-3cb5-41e5-8210-57ed2bd68746 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:23.911409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26701f3b-3cb5-41e5-8210-57ed2bd68746 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:23.921175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26701f3b-3cb5-41e5-8210-57ed2bd68746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-26701f3b-3cb5-41e5-8210-57ed2bd68746 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:23.925970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:23.927042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 101/210] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:23.928123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc516b2a-e9e9-4501-a01e-234748ddb3de None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:23.928930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc516b2a-e9e9-4501-a01e-234748ddb3de None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:23.928930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc516b2a-e9e9-4501-a01e-234748ddb3de None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:23.930752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 102/211] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:23.935292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc516b2a-e9e9-4501-a01e-234748ddb3de None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:23.943106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c88f52-c4b9-4331-ac21-b4156008ab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:23.943735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c88f52-c4b9-4331-ac21-b4156008ab02 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:23.944231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c88f52-c4b9-4331-ac21-b4156008ab02 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:23.944344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c88f52-c4b9-4331-ac21-b4156008ab02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:23.945368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c88f52-c4b9-4331-ac21-b4156008ab02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:23.946817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c88f52-c4b9-4331-ac21-b4156008ab02 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:23.951060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 110/212] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:23.951060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dec669-658c-427b-9757-40ebcaa8cff3 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:23.951060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dec669-658c-427b-9757-40ebcaa8cff3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:23.951060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dec669-658c-427b-9757-40ebcaa8cff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:23.967757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-04dec669-658c-427b-9757-40ebcaa8cff3 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:23.968926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-04dec669-658c-427b-9757-40ebcaa8cff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:24.031397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:24.033761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 111/213] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:24.034789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dec669-658c-427b-9757-40ebcaa8cff3 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:24.040147 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc516b2a-e9e9-4501-a01e-234748ddb3de None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:24.063879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:24.078479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b457-b442-4908-9b1a-7bef00cb2952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:24.079132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b457-b442-4908-9b1a-7bef00cb2952 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:24.079416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b457-b442-4908-9b1a-7bef00cb2952 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:24.079690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b457-b442-4908-9b1a-7bef00cb2952 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:24.080357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033b457-b442-4908-9b1a-7bef00cb2952 None admin] RBAC: Adding query filter params (name=member) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:24.080662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033b457-b442-4908-9b1a-7bef00cb2952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:24.082044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033b457-b442-4908-9b1a-7bef00cb2952 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:24.090102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 103/214] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:24.095662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b457-b442-4908-9b1a-7bef00cb2952 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:24.145082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dec669-658c-427b-9757-40ebcaa8cff3 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:24.180864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f47838-48ea-46e2-a43e-1b0915fde356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:24.181466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f47838-48ea-46e2-a43e-1b0915fde356 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:24.181740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f47838-48ea-46e2-a43e-1b0915fde356 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:24.182016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f47838-48ea-46e2-a43e-1b0915fde356 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:24.183037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f47838-48ea-46e2-a43e-1b0915fde356 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:24.184786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f47838-48ea-46e2-a43e-1b0915fde356 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:24.207392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 112/215] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:24 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:24.208290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b457-b442-4908-9b1a-7bef00cb2952 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:24.247085 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bd90e-5746-422e-bbb0-c4ccb64e2815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:24.247745 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bd90e-5746-422e-bbb0-c4ccb64e2815 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:24.248602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bd90e-5746-422e-bbb0-c4ccb64e2815 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:24.248874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bd90e-5746-422e-bbb0-c4ccb64e2815 None admin] PATH_INFO: `/v3/users/02c6d2a62bbf47fab705c7c59ed910b5` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:24.263877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5bd90e-5746-422e-bbb0-c4ccb64e2815 None admin] RBAC: Authorizing `identity:get_user(user_id=02c6d2a62bbf47fab705c7c59ed910b5)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:24.265378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5bd90e-5746-422e-bbb0-c4ccb64e2815 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:24.267371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 104/216] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:24 2022] GET /identity/v3/users/02c6d2a62bbf47fab705c7c59ed910b5 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:24.273583 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f47838-48ea-46e2-a43e-1b0915fde356 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:24.304617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bd90e-5746-422e-bbb0-c4ccb64e2815 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:24.329821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:24.373690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:24.400116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f47838-48ea-46e2-a43e-1b0915fde356 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:24.418927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bd90e-5746-422e-bbb0-c4ccb64e2815 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:24.443734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ae87a-6f3a-421d-b388-e6c27a230838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:24.444352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ae87a-6f3a-421d-b388-e6c27a230838 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:24.444661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ae87a-6f3a-421d-b388-e6c27a230838 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:24.444883 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ae87a-6f3a-421d-b388-e6c27a230838 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:24.446297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ae87a-6f3a-421d-b388-e6c27a230838 None admin] RBAC: Adding query filter params (role.id=3c56cfa91fba453da353f21d7d6c1189, user.id=02c6d2a62bbf47fab705c7c59ed910b5, scope.system=all) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:24.446297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ae87a-6f3a-421d-b388-e6c27a230838 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:24.447385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ae87a-6f3a-421d-b388-e6c27a230838 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:24.453295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 113/217] 10.222.0.58 () {56 vars in 1308 bytes} [Fri Oct 28 18:53:24 2022] GET /identity/v3/role_assignments?user.id=02c6d2a62bbf47fab705c7c59ed910b5&scope.system=all&role.id=3c56cfa91fba453da353f21d7d6c1189 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:24.459637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d5acb-2ca6-4d00-b8d3-35ab12df3071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:24.460307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d5acb-2ca6-4d00-b8d3-35ab12df3071 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:24.460588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d5acb-2ca6-4d00-b8d3-35ab12df3071 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:24.460860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d5acb-2ca6-4d00-b8d3-35ab12df3071 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:24.474502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d5acb-2ca6-4d00-b8d3-35ab12df3071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-2c8d5acb-2ca6-4d00-b8d3-35ab12df3071 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:24.480135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:24.481258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:24.482029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 105/218] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:24.489423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ae87a-6f3a-421d-b388-e6c27a230838 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:24.610349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ae87a-6f3a-421d-b388-e6c27a230838 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:24.647460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5339de-a17d-43f3-bb36-cd7b4d640eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:24.648073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5339de-a17d-43f3-bb36-cd7b4d640eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:24.648354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5339de-a17d-43f3-bb36-cd7b4d640eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:24.648629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5339de-a17d-43f3-bb36-cd7b4d640eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:24.649300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5339de-a17d-43f3-bb36-cd7b4d640eb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:24.649603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5339de-a17d-43f3-bb36-cd7b4d640eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:24.651014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5339de-a17d-43f3-bb36-cd7b4d640eb3 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:24.660815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 114/219] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:24.666543 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d5acb-2ca6-4d00-b8d3-35ab12df3071 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:24.766765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:24.783107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d5acb-2ca6-4d00-b8d3-35ab12df3071 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:24.819140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89f59-41e8-4033-8af0-6a425f8ad983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:24.819777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89f59-41e8-4033-8af0-6a425f8ad983 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:24.820056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89f59-41e8-4033-8af0-6a425f8ad983 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:24.820331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89f59-41e8-4033-8af0-6a425f8ad983 None admin] PATH_INFO: `/v3/users/fe029248c81c46329b912053bf2894bf` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:24.821484 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d89f59-41e8-4033-8af0-6a425f8ad983 None admin] RBAC: Authorizing `identity:get_user(user_id=fe029248c81c46329b912053bf2894bf)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:24.822933 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d89f59-41e8-4033-8af0-6a425f8ad983 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:24.824953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 106/220] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:24 2022] GET /identity/v3/users/fe029248c81c46329b912053bf2894bf => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:24.831068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5339de-a17d-43f3-bb36-cd7b4d640eb3 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:24.842888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:24.938537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5339de-a17d-43f3-bb36-cd7b4d640eb3 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:24.973843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778fac5-a689-4db0-a776-696e504facd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:24.974453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778fac5-a689-4db0-a776-696e504facd9 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:24.974734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778fac5-a689-4db0-a776-696e504facd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:24.975030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778fac5-a689-4db0-a776-696e504facd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:25.047094 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778fac5-a689-4db0-a776-696e504facd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:25.048567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778fac5-a689-4db0-a776-696e504facd9 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:25.059163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:25.157317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 115/221] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:25.162648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89f59-41e8-4033-8af0-6a425f8ad983 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:25.268059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89f59-41e8-4033-8af0-6a425f8ad983 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:25.304868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c95b47-c11c-4e62-a468-8991f92e1d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:25.305574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c95b47-c11c-4e62-a468-8991f92e1d08 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:25.305857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c95b47-c11c-4e62-a468-8991f92e1d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:25.306133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c95b47-c11c-4e62-a468-8991f92e1d08 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:25.307295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c95b47-c11c-4e62-a468-8991f92e1d08 None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:25.308742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c95b47-c11c-4e62-a468-8991f92e1d08 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:25.311216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 107/222] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:25 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:25.317139 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778fac5-a689-4db0-a776-696e504facd9 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:25.425160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778fac5-a689-4db0-a776-696e504facd9 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:25.434514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:25.460824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e10ed9-e472-46ba-8d19-51ba1f8c13f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:25.461430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e10ed9-e472-46ba-8d19-51ba1f8c13f2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:25.461731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e10ed9-e472-46ba-8d19-51ba1f8c13f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:25.462004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e10ed9-e472-46ba-8d19-51ba1f8c13f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:25.462719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e10ed9-e472-46ba-8d19-51ba1f8c13f2 None admin] RBAC: Adding query filter params (role.id=5981233f19954c69b900f0bbdbb84fd0, scope.project.id=2ccdf5ff122f47538ebe72016c0cab36, user.id=fe029248c81c46329b912053bf2894bf) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:25.463035 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e10ed9-e472-46ba-8d19-51ba1f8c13f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:25.464466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e10ed9-e472-46ba-8d19-51ba1f8c13f2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:25.470819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 116/223] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:25 2022] GET /identity/v3/role_assignments?user.id=fe029248c81c46329b912053bf2894bf&scope.project.id=2ccdf5ff122f47538ebe72016c0cab36&role.id=5981233f19954c69b900f0bbdbb84fd0 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:26.568926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57397-abfd-485c-9bb0-aa68501ad0e0 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:26.569450 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57397-abfd-485c-9bb0-aa68501ad0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:26.569450 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57397-abfd-485c-9bb0-aa68501ad0e0 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:26.571009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 108/224] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:26.576792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ebd74-83b7-48fc-b0a1-b36befa23d13 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:26.577195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ebd74-83b7-48fc-b0a1-b36befa23d13 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:26.577330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ebd74-83b7-48fc-b0a1-b36befa23d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:26.596398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-bb0ebd74-83b7-48fc-b0a1-b36befa23d13 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:26.597813 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-bb0ebd74-83b7-48fc-b0a1-b36befa23d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:26.617486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2b538-0298-4452-86fe-56705598e604 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:26.617766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2b538-0298-4452-86fe-56705598e604 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:26.618200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2b538-0298-4452-86fe-56705598e604 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:26.619554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 109/225] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:26.624566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9077336-260b-4257-b34a-124f34268f00 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:26.624847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9077336-260b-4257-b34a-124f34268f00 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:26.625115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9077336-260b-4257-b34a-124f34268f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:26.643964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a9077336-260b-4257-b34a-124f34268f00 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:26.645464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a9077336-260b-4257-b34a-124f34268f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:26.657993 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:26.668512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 117/226] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:26.669849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeed812-040b-400b-8dd9-4cd3d57f69a6 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:26.670136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeed812-040b-400b-8dd9-4cd3d57f69a6 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:26.670473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeed812-040b-400b-8dd9-4cd3d57f69a6 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:26.671690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 118/227] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:26.680318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb144e-42e3-4397-8db7-3b9ce0846e76 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:26.680593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb144e-42e3-4397-8db7-3b9ce0846e76 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:26.680857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb144e-42e3-4397-8db7-3b9ce0846e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:26.697903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-57fb144e-42e3-4397-8db7-3b9ce0846e76 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:26.699090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-57fb144e-42e3-4397-8db7-3b9ce0846e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:26.722579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 110/228] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:26.770890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 119/229] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:26.935953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9077336-260b-4257-b34a-124f34268f00 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:26.987784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb144e-42e3-4397-8db7-3b9ce0846e76 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:27.020654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:27.043381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9077336-260b-4257-b34a-124f34268f00 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.078770 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8a930-56f6-4929-a6b9-99e1a153ac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.079451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8a930-56f6-4929-a6b9-99e1a153ac6a None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.079732 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8a930-56f6-4929-a6b9-99e1a153ac6a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.080006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8a930-56f6-4929-a6b9-99e1a153ac6a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.090100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8a930-56f6-4929-a6b9-99e1a153ac6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:27.094918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb144e-42e3-4397-8db7-3b9ce0846e76 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-f5f8a930-56f6-4929-a6b9-99e1a153ac6a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:27.095175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:27.096169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 111/230] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:27.097201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8a930-56f6-4929-a6b9-99e1a153ac6a None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:27.106746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:27.132659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0f96f-df07-4841-8e5d-a1357e6852b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.133288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0f96f-df07-4841-8e5d-a1357e6852b6 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.133565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0f96f-df07-4841-8e5d-a1357e6852b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.133836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0f96f-df07-4841-8e5d-a1357e6852b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.143921 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0f96f-df07-4841-8e5d-a1357e6852b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-20a0f96f-df07-4841-8e5d-a1357e6852b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:27.148739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:27.149622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 120/231] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:27.152274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0f96f-df07-4841-8e5d-a1357e6852b6 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:27.198477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8a930-56f6-4929-a6b9-99e1a153ac6a None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.231329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ba762-1b19-43fa-abe7-2d3d2e08d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.231958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ba762-1b19-43fa-abe7-2d3d2e08d1fa None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.232810 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ba762-1b19-43fa-abe7-2d3d2e08d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.233084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ba762-1b19-43fa-abe7-2d3d2e08d1fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.234187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ba762-1b19-43fa-abe7-2d3d2e08d1fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:27.235677 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ba762-1b19-43fa-abe7-2d3d2e08d1fa None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:27.237225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 112/232] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:27.237981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ba762-1b19-43fa-abe7-2d3d2e08d1fa None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:27.242082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:27.252520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0f96f-df07-4841-8e5d-a1357e6852b6 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.286381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d44dea-cde5-4b34-9d78-ba9cf4348e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.286996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d44dea-cde5-4b34-9d78-ba9cf4348e0a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.287286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d44dea-cde5-4b34-9d78-ba9cf4348e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.287551 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d44dea-cde5-4b34-9d78-ba9cf4348e0a None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.288203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d44dea-cde5-4b34-9d78-ba9cf4348e0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:27.288501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d44dea-cde5-4b34-9d78-ba9cf4348e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:27.289875 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d44dea-cde5-4b34-9d78-ba9cf4348e0a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:27.302776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 121/233] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:27.302776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d44dea-cde5-4b34-9d78-ba9cf4348e0a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:27.313591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:27.338415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ba762-1b19-43fa-abe7-2d3d2e08d1fa None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.371927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08368262-804e-4d48-9d76-f4071f2a4409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.372532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08368262-804e-4d48-9d76-f4071f2a4409 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.373365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08368262-804e-4d48-9d76-f4071f2a4409 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.373650 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08368262-804e-4d48-9d76-f4071f2a4409 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.374312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08368262-804e-4d48-9d76-f4071f2a4409 None admin] RBAC: Adding query filter params (name=member) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:27.374614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08368262-804e-4d48-9d76-f4071f2a4409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:27.376026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08368262-804e-4d48-9d76-f4071f2a4409 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:27.383997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 113/234] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:27.384783 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08368262-804e-4d48-9d76-f4071f2a4409 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:27.400707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d44dea-cde5-4b34-9d78-ba9cf4348e0a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.434188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb80563-b65a-4e08-a812-e0845340f89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.434788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb80563-b65a-4e08-a812-e0845340f89c None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.435113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb80563-b65a-4e08-a812-e0845340f89c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.435924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb80563-b65a-4e08-a812-e0845340f89c None admin] PATH_INFO: `/v3/groups` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.436538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb80563-b65a-4e08-a812-e0845340f89c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:27.437960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb80563-b65a-4e08-a812-e0845340f89c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:27.463164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 122/235] 10.222.0.58 () {60 vars in 1134 bytes} [Fri Oct 28 18:53:27 2022] POST /identity/v3/groups => generated 229 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:27.463164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb80563-b65a-4e08-a812-e0845340f89c None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:27.470406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:27.489354 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08368262-804e-4d48-9d76-f4071f2a4409 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.526454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ae3e2-f1f5-4312-995b-2f0834a66ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.527127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ae3e2-f1f5-4312-995b-2f0834a66ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.527414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ae3e2-f1f5-4312-995b-2f0834a66ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.527679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ae3e2-f1f5-4312-995b-2f0834a66ae2 None admin] PATH_INFO: `/v3/users/44d73cd242f14b9eb4585201908dc63a` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.531625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:27.545765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ae3e2-f1f5-4312-995b-2f0834a66ae2 None admin] RBAC: Authorizing `identity:get_user(user_id=44d73cd242f14b9eb4585201908dc63a)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:27.547906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ae3e2-f1f5-4312-995b-2f0834a66ae2 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:27.550227 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 114/236] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:27 2022] GET /identity/v3/users/44d73cd242f14b9eb4585201908dc63a => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:27.556068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ae3e2-f1f5-4312-995b-2f0834a66ae2 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:27.575639 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb80563-b65a-4e08-a812-e0845340f89c None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.614104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8272e8-918b-49df-97d1-48c5e90f0568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.615359 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8272e8-918b-49df-97d1-48c5e90f0568 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.615645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8272e8-918b-49df-97d1-48c5e90f0568 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.615942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8272e8-918b-49df-97d1-48c5e90f0568 None admin] PATH_INFO: `/v3/users/02c6d2a62bbf47fab705c7c59ed910b5` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.617197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8272e8-918b-49df-97d1-48c5e90f0568 None admin] RBAC: Authorizing `identity:get_user(user_id=02c6d2a62bbf47fab705c7c59ed910b5)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:27.618693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8272e8-918b-49df-97d1-48c5e90f0568 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:27.620363 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 123/237] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:27 2022] GET /identity/v3/users/02c6d2a62bbf47fab705c7c59ed910b5 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:27.626234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8272e8-918b-49df-97d1-48c5e90f0568 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:27.672702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ae3e2-f1f5-4312-995b-2f0834a66ae2 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.707480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a91c8-223e-4df7-94e9-47971788ef24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.708089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a91c8-223e-4df7-94e9-47971788ef24 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.708363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a91c8-223e-4df7-94e9-47971788ef24 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.708658 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a91c8-223e-4df7-94e9-47971788ef24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.709378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a91c8-223e-4df7-94e9-47971788ef24 None admin] RBAC: Adding query filter params (role.id=5981233f19954c69b900f0bbdbb84fd0, user.id=44d73cd242f14b9eb4585201908dc63a, scope.system=all) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:27.709673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a91c8-223e-4df7-94e9-47971788ef24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:27.711181 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a91c8-223e-4df7-94e9-47971788ef24 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:27.717482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 115/238] 10.222.0.58 () {56 vars in 1308 bytes} [Fri Oct 28 18:53:27 2022] GET /identity/v3/role_assignments?user.id=44d73cd242f14b9eb4585201908dc63a&scope.system=all&role.id=5981233f19954c69b900f0bbdbb84fd0 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:27.727525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:27.754358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8272e8-918b-49df-97d1-48c5e90f0568 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:27.793598 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cd02b-5ca4-46b3-b3c4-27204f25ccb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:27.794266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cd02b-5ca4-46b3-b3c4-27204f25ccb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:27.794548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cd02b-5ca4-46b3-b3c4-27204f25ccb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:27.795460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cd02b-5ca4-46b3-b3c4-27204f25ccb0 None admin] PATH_INFO: `/v3/system/users/02c6d2a62bbf47fab705c7c59ed910b5/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:27.796988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cd02b-5ca4-46b3-b3c4-27204f25ccb0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=02c6d2a62bbf47fab705c7c59ed910b5, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:27.798421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cd02b-5ca4-46b3-b3c4-27204f25ccb0 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:27.810057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 124/239] 10.222.0.58 () {58 vars in 1334 bytes} [Fri Oct 28 18:53:27 2022] PUT /identity/v3/system/users/02c6d2a62bbf47fab705c7c59ed910b5/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:29.637891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2818620-c97d-4945-aa70-0cfede972d08 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:29.638445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2818620-c97d-4945-aa70-0cfede972d08 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:29.638521 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2818620-c97d-4945-aa70-0cfede972d08 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:29.640187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 116/240] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:29.645338 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75332687-68a9-477d-9be6-5a5a83d1ab72 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:29.645739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75332687-68a9-477d-9be6-5a5a83d1ab72 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:29.645917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75332687-68a9-477d-9be6-5a5a83d1ab72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:29.664739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-75332687-68a9-477d-9be6-5a5a83d1ab72 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:29.665914 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-75332687-68a9-477d-9be6-5a5a83d1ab72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:29.737341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 125/241] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:29.848040 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc58a2-3abe-4099-a96c-41752d3d2947 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:29.848322 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc58a2-3abe-4099-a96c-41752d3d2947 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:29.848778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc58a2-3abe-4099-a96c-41752d3d2947 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:29.850177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 117/242] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:29.856191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015878d2-8b18-47be-b8a9-e20b5e4e17d5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:29.856473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015878d2-8b18-47be-b8a9-e20b5e4e17d5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:29.856918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015878d2-8b18-47be-b8a9-e20b5e4e17d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:29.875887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-015878d2-8b18-47be-b8a9-e20b5e4e17d5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:29.877070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-015878d2-8b18-47be-b8a9-e20b5e4e17d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:29.942871 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 126/243] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:29.953313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea76cd-41d6-4745-8fcd-3e86baeb1d0a None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:29.953805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea76cd-41d6-4745-8fcd-3e86baeb1d0a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:29.954137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea76cd-41d6-4745-8fcd-3e86baeb1d0a None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:29.955564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 118/244] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:29.960653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698914bd-e43d-4a77-bd2a-57816d2a3fcc None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:29.960944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698914bd-e43d-4a77-bd2a-57816d2a3fcc None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:29.961219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698914bd-e43d-4a77-bd2a-57816d2a3fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:29.974133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:29.980881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-698914bd-e43d-4a77-bd2a-57816d2a3fcc None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:29.982787 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-698914bd-e43d-4a77-bd2a-57816d2a3fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:30.003988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea76cd-41d6-4745-8fcd-3e86baeb1d0a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:30.055049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 127/245] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:30.067345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:30.110813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea76cd-41d6-4745-8fcd-3e86baeb1d0a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:30.144220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76029402-4b87-469c-ba4a-0c4936b04f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:30.144845 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76029402-4b87-469c-ba4a-0c4936b04f91 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:30.145123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76029402-4b87-469c-ba4a-0c4936b04f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:30.145384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76029402-4b87-469c-ba4a-0c4936b04f91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:30.146455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76029402-4b87-469c-ba4a-0c4936b04f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:30.147912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76029402-4b87-469c-ba4a-0c4936b04f91 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:30.149601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 119/246] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:30.155530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698914bd-e43d-4a77-bd2a-57816d2a3fcc None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:30.211934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76029402-4b87-469c-ba4a-0c4936b04f91 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:30.254934 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698914bd-e43d-4a77-bd2a-57816d2a3fcc None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:30.290078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0b875-76e3-42d4-ac2c-2118e2247bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:30.290681 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0b875-76e3-42d4-ac2c-2118e2247bb0 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:30.290954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0b875-76e3-42d4-ac2c-2118e2247bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:30.291250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0b875-76e3-42d4-ac2c-2118e2247bb0 None admin] PATH_INFO: `/v3/groups` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:30.291859 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0b875-76e3-42d4-ac2c-2118e2247bb0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:30.293289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0b875-76e3-42d4-ac2c-2118e2247bb0 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:30.297994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:30.311406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 128/247] 10.222.0.58 () {60 vars in 1134 bytes} [Fri Oct 28 18:53:30 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:53:30.320318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76029402-4b87-469c-ba4a-0c4936b04f91 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:30.334390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0b875-76e3-42d4-ac2c-2118e2247bb0 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:30.354791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:30.364058 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119187-b273-4732-b0d2-fb4fcc1517f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:30.364901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119187-b273-4732-b0d2-fb4fcc1517f9 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:30.365195 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119187-b273-4732-b0d2-fb4fcc1517f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:30.365458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119187-b273-4732-b0d2-fb4fcc1517f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:30.379093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d119187-b273-4732-b0d2-fb4fcc1517f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-9d119187-b273-4732-b0d2-fb4fcc1517f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:30.384772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:30.385744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:30.386546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:30.391731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 120/248] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:30.392692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119187-b273-4732-b0d2-fb4fcc1517f9 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:30.457023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0b875-76e3-42d4-ac2c-2118e2247bb0 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:30.495103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49727040-8d01-4d07-972a-107832d297aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:30.495751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49727040-8d01-4d07-972a-107832d297aa None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:30.496029 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49727040-8d01-4d07-972a-107832d297aa None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:30.496301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49727040-8d01-4d07-972a-107832d297aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:30.507546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49727040-8d01-4d07-972a-107832d297aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-49727040-8d01-4d07-972a-107832d297aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:30.512963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:30.514423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:30.515851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 129/249] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:30.519026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119187-b273-4732-b0d2-fb4fcc1517f9 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:30.520711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49727040-8d01-4d07-972a-107832d297aa None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:30.564705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57abbf4-37a7-434d-8bcd-62d9b181c0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:30.565323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57abbf4-37a7-434d-8bcd-62d9b181c0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:30.565599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57abbf4-37a7-434d-8bcd-62d9b181c0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:30.565874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57abbf4-37a7-434d-8bcd-62d9b181c0c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:30.566538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57abbf4-37a7-434d-8bcd-62d9b181c0c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:30.566841 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57abbf4-37a7-434d-8bcd-62d9b181c0c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:30.568259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57abbf4-37a7-434d-8bcd-62d9b181c0c4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:30.579551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 121/250] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:30.586364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57abbf4-37a7-434d-8bcd-62d9b181c0c4 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:30.588411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:30.658449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:30.668990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49727040-8d01-4d07-972a-107832d297aa None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:30.729191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874676f-27af-457e-a34f-822046e7a4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:30.729803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874676f-27af-457e-a34f-822046e7a4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:30.730080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874676f-27af-457e-a34f-822046e7a4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:30.730348 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874676f-27af-457e-a34f-822046e7a4a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:30.731250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874676f-27af-457e-a34f-822046e7a4a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:30.731563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874676f-27af-457e-a34f-822046e7a4a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:30.733186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874676f-27af-457e-a34f-822046e7a4a4 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:30.752043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 130/251] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:30.752043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874676f-27af-457e-a34f-822046e7a4a4 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:30.767427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57abbf4-37a7-434d-8bcd-62d9b181c0c4 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:30.819392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c47f90-751a-460d-9095-993f853ffb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:30.821793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c47f90-751a-460d-9095-993f853ffb2c None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:30.821793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c47f90-751a-460d-9095-993f853ffb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:30.821793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c47f90-751a-460d-9095-993f853ffb2c None admin] PATH_INFO: `/v3/users/44d73cd242f14b9eb4585201908dc63a` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:30.822012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c47f90-751a-460d-9095-993f853ffb2c None admin] RBAC: Authorizing `identity:get_user(user_id=44d73cd242f14b9eb4585201908dc63a)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:30.823962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c47f90-751a-460d-9095-993f853ffb2c None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:30.825086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 122/252] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:30 2022] GET /identity/v3/users/44d73cd242f14b9eb4585201908dc63a => generated 293 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:30.832916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c47f90-751a-460d-9095-993f853ffb2c None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:30.894179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874676f-27af-457e-a34f-822046e7a4a4 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:30.932851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:30.971253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16fc0a-e190-47f1-9856-53e870b2477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:30.972183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16fc0a-e190-47f1-9856-53e870b2477d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:30.972651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16fc0a-e190-47f1-9856-53e870b2477d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:30.973089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16fc0a-e190-47f1-9856-53e870b2477d None admin] PATH_INFO: `/v3/users/02c6d2a62bbf47fab705c7c59ed910b5` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:30.974865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16fc0a-e190-47f1-9856-53e870b2477d None admin] RBAC: Authorizing `identity:get_user(user_id=02c6d2a62bbf47fab705c7c59ed910b5)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:30.976519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16fc0a-e190-47f1-9856-53e870b2477d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:30.979606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 131/253] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:30 2022] GET /identity/v3/users/02c6d2a62bbf47fab705c7c59ed910b5 => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:30.985691 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16fc0a-e190-47f1-9856-53e870b2477d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:31.027388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c47f90-751a-460d-9095-993f853ffb2c None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:31.062351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:31.095573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9f54c-4360-4d02-86d7-ef5be0d65007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:31.096252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9f54c-4360-4d02-86d7-ef5be0d65007 None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:31.096534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9f54c-4360-4d02-86d7-ef5be0d65007 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:31.096811 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9f54c-4360-4d02-86d7-ef5be0d65007 None admin] PATH_INFO: `/v3/system/users/44d73cd242f14b9eb4585201908dc63a/roles/5981233f19954c69b900f0bbdbb84fd0` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:31.098804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9f54c-4360-4d02-86d7-ef5be0d65007 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=44d73cd242f14b9eb4585201908dc63a, role_id=5981233f19954c69b900f0bbdbb84fd0)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:31.100401 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9f54c-4360-4d02-86d7-ef5be0d65007 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:31.122736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 123/254] 10.222.0.58 () {58 vars in 1334 bytes} [Fri Oct 28 18:53:30 2022] PUT /identity/v3/system/users/44d73cd242f14b9eb4585201908dc63a/roles/5981233f19954c69b900f0bbdbb84fd0 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:31.164111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16fc0a-e190-47f1-9856-53e870b2477d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:31.206129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4e825-cec4-4cf8-8459-591863905330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:31.206767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4e825-cec4-4cf8-8459-591863905330 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:31.207076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4e825-cec4-4cf8-8459-591863905330 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:31.207356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4e825-cec4-4cf8-8459-591863905330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:31.208099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4e825-cec4-4cf8-8459-591863905330 None admin] RBAC: Adding query filter params (role.id=3c56cfa91fba453da353f21d7d6c1189, user.id=02c6d2a62bbf47fab705c7c59ed910b5, scope.system=all) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:31.208464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4e825-cec4-4cf8-8459-591863905330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:31.209936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4e825-cec4-4cf8-8459-591863905330 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:31.223783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 132/255] 10.222.0.58 () {56 vars in 1308 bytes} [Fri Oct 28 18:53:30 2022] GET /identity/v3/role_assignments?user.id=02c6d2a62bbf47fab705c7c59ed910b5&scope.system=all&role.id=3c56cfa91fba453da353f21d7d6c1189 => generated 510 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:32.776409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc664fc3-38e5-499a-9da1-6a237de58084 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:32.776919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc664fc3-38e5-499a-9da1-6a237de58084 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:32.776999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc664fc3-38e5-499a-9da1-6a237de58084 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:32.778440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 124/256] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:32.789518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec5648-4bdb-410c-ae99-c5519c306a6b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:32.789940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec5648-4bdb-410c-ae99-c5519c306a6b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:32.790133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec5648-4bdb-410c-ae99-c5519c306a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:32.816434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-cdec5648-4bdb-410c-ae99-c5519c306a6b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:32.817649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-cdec5648-4bdb-410c-ae99-c5519c306a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:32.872905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:32.889138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 133/257] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 28 18:53:33.220329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc664fc3-38e5-499a-9da1-6a237de58084 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:33.236621 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc664fc3-38e5-499a-9da1-6a237de58084 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:33.241144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9a685-5c5a-4a7a-9824-2fb4a7f894ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:33.241779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9a685-5c5a-4a7a-9824-2fb4a7f894ec admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:33.242057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9a685-5c5a-4a7a-9824-2fb4a7f894ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:33.242331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9a685-5c5a-4a7a-9824-2fb4a7f894ec admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:33.260486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9a685-5c5a-4a7a-9824-2fb4a7f894ec admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-eab9a685-5c5a-4a7a-9824-2fb4a7f894ec admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:33.265964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:33.266818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:33.267626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 125/258] 10.222.0.58 () {56 vars in 1120 bytes} [Fri Oct 28 18:53:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:33.274537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec5648-4bdb-410c-ae99-c5519c306a6b None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:33.289314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec5648-4bdb-410c-ae99-c5519c306a6b None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:33.298461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:33.299115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:33.299396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:33.299665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:33.300347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:33.300649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:33.302447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING py.warnings [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 28 18:53:33.302447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: warnings.warn(msg) Oct 28 18:53:33.302447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: Oct 28 18:53:33.303420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:33.319364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 134/259] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:53:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:33.325247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9a685-5c5a-4a7a-9824-2fb4a7f894ec admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:33.335975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9a685-5c5a-4a7a-9824-2fb4a7f894ec admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:33.347519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8604f40-be57-4a9c-9294-fea2dcb8005e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:33.348171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8604f40-be57-4a9c-9294-fea2dcb8005e admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:33.348453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8604f40-be57-4a9c-9294-fea2dcb8005e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:33.348729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8604f40-be57-4a9c-9294-fea2dcb8005e admin admin] PATH_INFO: `/v3/groups/9a1e5f068a32416cbe47d515075df556` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:33.367691 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8604f40-be57-4a9c-9294-fea2dcb8005e admin admin] RBAC: Authorizing `identity:get_group(group_id=9a1e5f068a32416cbe47d515075df556)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:33.369765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING py.warnings [None req-e8604f40-be57-4a9c-9294-fea2dcb8005e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 18:53:33.369765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: warnings.warn(msg) Oct 28 18:53:33.369765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: Oct 28 18:53:33.371123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8604f40-be57-4a9c-9294-fea2dcb8005e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:33.373489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 126/260] 10.222.0.58 () {56 vars in 1201 bytes} [Fri Oct 28 18:53:33 2022] GET /identity/v3/groups/9a1e5f068a32416cbe47d515075df556 => generated 226 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:33.378972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:33.396509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb966709-d4eb-4fc2-b658-33685fe16ff2 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:33.403037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fca06-26c0-4564-844f-c4f39aa69a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:33.403648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fca06-26c0-4564-844f-c4f39aa69a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:33.404530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fca06-26c0-4564-844f-c4f39aa69a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:33.404803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fca06-26c0-4564-844f-c4f39aa69a16 admin admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:33.406014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fca06-26c0-4564-844f-c4f39aa69a16 admin admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:33.412193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fca06-26c0-4564-844f-c4f39aa69a16 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:33.414332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 135/261] 10.222.0.58 () {56 vars in 1207 bytes} [Fri Oct 28 18:53:33 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:33.420104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8604f40-be57-4a9c-9294-fea2dcb8005e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:33.430346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8604f40-be57-4a9c-9294-fea2dcb8005e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:33.438812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e876248-9a6f-4f29-ba72-ea71fc2daf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:33.439530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e876248-9a6f-4f29-ba72-ea71fc2daf0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:33.439806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e876248-9a6f-4f29-ba72-ea71fc2daf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:33.440109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e876248-9a6f-4f29-ba72-ea71fc2daf0a admin admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652/groups/9a1e5f068a32416cbe47d515075df556/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:33.444112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e876248-9a6f-4f29-ba72-ea71fc2daf0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c771455a484eb080ab85c17582f652, group_id=9a1e5f068a32416cbe47d515075df556, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:33.445136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING py.warnings [None req-9e876248-9a6f-4f29-ba72-ea71fc2daf0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 18:53:33.445136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: warnings.warn(msg) Oct 28 18:53:33.445136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: Oct 28 18:53:33.447147 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e876248-9a6f-4f29-ba72-ea71fc2daf0a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:33.478780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 127/262] 10.222.0.58 () {58 vars in 1463 bytes} [Fri Oct 28 18:53:33 2022] PUT /identity/v3/projects/96c771455a484eb080ab85c17582f652/groups/9a1e5f068a32416cbe47d515075df556/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:33.517046 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3c673-e15a-4395-b789-8f0770fab00b None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:33.517367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3c673-e15a-4395-b789-8f0770fab00b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:33.517585 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3c673-e15a-4395-b789-8f0770fab00b None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:33.519292 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 136/263] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:33.524741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26616eb-edee-4cc9-bfa1-fb348eb212e8 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:33.525026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26616eb-edee-4cc9-bfa1-fb348eb212e8 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:33.525298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26616eb-edee-4cc9-bfa1-fb348eb212e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:33.549442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-d26616eb-edee-4cc9-bfa1-fb348eb212e8 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:33.550686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d26616eb-edee-4cc9-bfa1-fb348eb212e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:33.621129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 128/264] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 28 18:53:33.765445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fc35d-56ba-4bf0-8294-2ead396eb4dc None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:33.765779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fc35d-56ba-4bf0-8294-2ead396eb4dc None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:33.765990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fc35d-56ba-4bf0-8294-2ead396eb4dc None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:33.767635 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 137/265] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:33.773945 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6a642-01b4-4e91-8e18-2f9afa5a9cd0 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:33.774243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6a642-01b4-4e91-8e18-2f9afa5a9cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:33.774494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6a642-01b4-4e91-8e18-2f9afa5a9cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:33.802114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-c3b6a642-01b4-4e91-8e18-2f9afa5a9cd0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:33.803353 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c3b6a642-01b4-4e91-8e18-2f9afa5a9cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:33.814066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 129/266] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 28 18:53:34.151178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fc35d-56ba-4bf0-8294-2ead396eb4dc None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.168759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fc35d-56ba-4bf0-8294-2ead396eb4dc None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.173843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682ee87-6e37-427d-97d0-611fd0af66bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.174478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682ee87-6e37-427d-97d0-611fd0af66bf admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.174755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682ee87-6e37-427d-97d0-611fd0af66bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.175048 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682ee87-6e37-427d-97d0-611fd0af66bf admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.193120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682ee87-6e37-427d-97d0-611fd0af66bf admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:34.197681 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6a642-01b4-4e91-8e18-2f9afa5a9cd0 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-b682ee87-6e37-427d-97d0-611fd0af66bf admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:34.198724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:34.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:34.200382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 138/267] 10.222.0.58 () {56 vars in 1120 bytes} [Fri Oct 28 18:53:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:34.208342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682ee87-6e37-427d-97d0-611fd0af66bf admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.212737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6a642-01b4-4e91-8e18-2f9afa5a9cd0 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.218755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682ee87-6e37-427d-97d0-611fd0af66bf admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.222780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a0043-8f56-4980-8f3a-d6dec1cc515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.223315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf06ab-62bc-4c26-91fb-bb969d74dd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.223525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a0043-8f56-4980-8f3a-d6dec1cc515d admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.223730 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a0043-8f56-4980-8f3a-d6dec1cc515d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.223909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf06ab-62bc-4c26-91fb-bb969d74dd8e admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.224054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a0043-8f56-4980-8f3a-d6dec1cc515d admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.224196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf06ab-62bc-4c26-91fb-bb969d74dd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.224442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf06ab-62bc-4c26-91fb-bb969d74dd8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.225110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf06ab-62bc-4c26-91fb-bb969d74dd8e admin admin] RBAC: Adding query filter params (name=member) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:34.225409 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf06ab-62bc-4c26-91fb-bb969d74dd8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:34.227193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf06ab-62bc-4c26-91fb-bb969d74dd8e admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:34.240335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 139/268] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:53:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:34.244164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a0043-8f56-4980-8f3a-d6dec1cc515d admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:34.246417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf06ab-62bc-4c26-91fb-bb969d74dd8e admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.260353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf06ab-62bc-4c26-91fb-bb969d74dd8e admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-856a0043-8f56-4980-8f3a-d6dec1cc515d admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:34.261867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:34.263955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:34.265847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 130/269] 10.222.0.58 () {56 vars in 1135 bytes} [Fri Oct 28 18:53:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 66 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:34.267456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575847-dd80-4f00-a503-74bb6e0c4287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.269707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575847-dd80-4f00-a503-74bb6e0c4287 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.269908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a0043-8f56-4980-8f3a-d6dec1cc515d admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.274046 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575847-dd80-4f00-a503-74bb6e0c4287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.274344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575847-dd80-4f00-a503-74bb6e0c4287 admin admin] PATH_INFO: `/v3/groups/9a1e5f068a32416cbe47d515075df556` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.277367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a575847-dd80-4f00-a503-74bb6e0c4287 admin admin] RBAC: Authorizing `identity:get_group(group_id=9a1e5f068a32416cbe47d515075df556)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:34.278421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING py.warnings [None req-2a575847-dd80-4f00-a503-74bb6e0c4287 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 18:53:34.278421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: warnings.warn(msg) Oct 28 18:53:34.278421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: Oct 28 18:53:34.280885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a575847-dd80-4f00-a503-74bb6e0c4287 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:34.283695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 140/270] 10.222.0.58 () {56 vars in 1201 bytes} [Fri Oct 28 18:53:34 2022] GET /identity/v3/groups/9a1e5f068a32416cbe47d515075df556 => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:34.285804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a0043-8f56-4980-8f3a-d6dec1cc515d admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.287486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575847-dd80-4f00-a503-74bb6e0c4287 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.296645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.297215 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.297491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.297764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.298496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:34.298797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:34.299885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING py.warnings [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 28 18:53:34.299885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: warnings.warn(msg) Oct 28 18:53:34.299885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: Oct 28 18:53:34.303030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:34.307919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575847-dd80-4f00-a503-74bb6e0c4287 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.318569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abe317-53df-4ab9-b1f9-947c25d976c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.320138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 131/271] 10.222.0.58 () {56 vars in 1132 bytes} [Fri Oct 28 18:53:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:34.320366 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abe317-53df-4ab9-b1f9-947c25d976c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.320653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abe317-53df-4ab9-b1f9-947c25d976c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.320977 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abe317-53df-4ab9-b1f9-947c25d976c7 admin admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.322230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abe317-53df-4ab9-b1f9-947c25d976c7 admin admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:34.324607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abe317-53df-4ab9-b1f9-947c25d976c7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:34.325434 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.332418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 141/272] 10.222.0.58 () {56 vars in 1207 bytes} [Fri Oct 28 18:53:34 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:34.336592 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abe317-53df-4ab9-b1f9-947c25d976c7 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.344153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eb8ea-49a1-4f04-ab85-fb2978d8e326 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.349200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abe317-53df-4ab9-b1f9-947c25d976c7 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.350422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4845f8b-1f77-4171-aeb5-ddbf9dbb15ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.351498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4845f8b-1f77-4171-aeb5-ddbf9dbb15ae admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.351915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4845f8b-1f77-4171-aeb5-ddbf9dbb15ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.352410 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4845f8b-1f77-4171-aeb5-ddbf9dbb15ae admin admin] PATH_INFO: `/v3/groups/9a1e5f068a32416cbe47d515075df556` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.359000 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4845f8b-1f77-4171-aeb5-ddbf9dbb15ae admin admin] RBAC: Authorizing `identity:get_group(group_id=9a1e5f068a32416cbe47d515075df556)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:34.359138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea801c93-ba5f-4a0f-9eb9-3a3815aad17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.359138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea801c93-ba5f-4a0f-9eb9-3a3815aad17b admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.359138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea801c93-ba5f-4a0f-9eb9-3a3815aad17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.359138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea801c93-ba5f-4a0f-9eb9-3a3815aad17b admin admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/groups/9a1e5f068a32416cbe47d515075df556/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.359886 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4845f8b-1f77-4171-aeb5-ddbf9dbb15ae admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:34.362788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea801c93-ba5f-4a0f-9eb9-3a3815aad17b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccdf5ff122f47538ebe72016c0cab36, group_id=9a1e5f068a32416cbe47d515075df556, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:34.363938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING py.warnings [None req-ea801c93-ba5f-4a0f-9eb9-3a3815aad17b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 18:53:34.363938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: warnings.warn(msg) Oct 28 18:53:34.363938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: Oct 28 18:53:34.364552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 132/273] 10.222.0.58 () {56 vars in 1201 bytes} [Fri Oct 28 18:53:34 2022] GET /identity/v3/groups/9a1e5f068a32416cbe47d515075df556 => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:34.366635 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea801c93-ba5f-4a0f-9eb9-3a3815aad17b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:34.369552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4845f8b-1f77-4171-aeb5-ddbf9dbb15ae admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.383757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4845f8b-1f77-4171-aeb5-ddbf9dbb15ae admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.394087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 142/274] 10.222.0.58 () {58 vars in 1463 bytes} [Fri Oct 28 18:53:34 2022] PUT /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/groups/9a1e5f068a32416cbe47d515075df556/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:34.396823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1fc19-5e2f-424c-a0a9-2269d27b533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.397582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1fc19-5e2f-424c-a0a9-2269d27b533a admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.397862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1fc19-5e2f-424c-a0a9-2269d27b533a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.398137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1fc19-5e2f-424c-a0a9-2269d27b533a admin admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.399428 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1fc19-5e2f-424c-a0a9-2269d27b533a admin admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:34.401917 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1fc19-5e2f-424c-a0a9-2269d27b533a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:34.404602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 133/275] 10.222.0.58 () {56 vars in 1207 bytes} [Fri Oct 28 18:53:34 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:34.410148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea801c93-ba5f-4a0f-9eb9-3a3815aad17b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:34.464824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07437bac-c722-4bb2-94c4-a57ed6737060 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.465155 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07437bac-c722-4bb2-94c4-a57ed6737060 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.465438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07437bac-c722-4bb2-94c4-a57ed6737060 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.467402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 134/276] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:34.474062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a122c7-3078-4081-b673-bbcc4298f95a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.474370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a122c7-3078-4081-b673-bbcc4298f95a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.474689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a122c7-3078-4081-b673-bbcc4298f95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.505322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea801c93-ba5f-4a0f-9eb9-3a3815aad17b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:34.511110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-b8a122c7-3078-4081-b673-bbcc4298f95a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:34.512533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b8a122c7-3078-4081-b673-bbcc4298f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:34.516219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb72c7-fca1-405a-b13f-c35fecfa1063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:34.521983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb72c7-fca1-405a-b13f-c35fecfa1063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:34.522577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb72c7-fca1-405a-b13f-c35fecfa1063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:34.523116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb72c7-fca1-405a-b13f-c35fecfa1063 admin admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652/groups/9a1e5f068a32416cbe47d515075df556/roles/0481df198a1a42f0a00358e0b9a938c5` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:34.525712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb72c7-fca1-405a-b13f-c35fecfa1063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c771455a484eb080ab85c17582f652, group_id=9a1e5f068a32416cbe47d515075df556, role_id=0481df198a1a42f0a00358e0b9a938c5)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:34.530010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb72c7-fca1-405a-b13f-c35fecfa1063 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:34.564742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 143/277] 10.222.0.58 () {58 vars in 1463 bytes} [Fri Oct 28 18:53:34 2022] PUT /identity/v3/projects/96c771455a484eb080ab85c17582f652/groups/9a1e5f068a32416cbe47d515075df556/roles/0481df198a1a42f0a00358e0b9a938c5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:34.609036 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:34.630510 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 135/278] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:35.228253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb72c7-fca1-405a-b13f-c35fecfa1063 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:35.251729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8b158-39c0-4bf5-8729-c6a28f7e430c None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:35.252233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8b158-39c0-4bf5-8729-c6a28f7e430c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:35.252335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8b158-39c0-4bf5-8729-c6a28f7e430c None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:35.253851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 136/279] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:35.259217 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eeed9-2914-449f-9630-b23a33f89573 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:35.259504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eeed9-2914-449f-9630-b23a33f89573 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:35.259781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eeed9-2914-449f-9630-b23a33f89573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:35.286738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-900eeed9-2914-449f-9630-b23a33f89573 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:35.287976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-900eeed9-2914-449f-9630-b23a33f89573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:35.333515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:35.379710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb72c7-fca1-405a-b13f-c35fecfa1063 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:35.381855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 137/280] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 28 18:53:35.436332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4f582-ae80-4977-a567-602c983a9c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:35.437342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4f582-ae80-4977-a567-602c983a9c68 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:35.437828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4f582-ae80-4977-a567-602c983a9c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:35.438287 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4f582-ae80-4977-a567-602c983a9c68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:35.455862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4f582-ae80-4977-a567-602c983a9c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-1ff4f582-ae80-4977-a567-602c983a9c68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:35.461488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:35.462328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 18:53:35.463129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:35.465599 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 144/281] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:35.472142 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900eeed9-2914-449f-9630-b23a33f89573 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:35.605189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900eeed9-2914-449f-9630-b23a33f89573 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:35.648568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc260016-3c1f-4d67-b909-c1635b5f2892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:35.649190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc260016-3c1f-4d67-b909-c1635b5f2892 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:35.649468 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc260016-3c1f-4d67-b909-c1635b5f2892 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:35.649737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc260016-3c1f-4d67-b909-c1635b5f2892 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:35.650415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc260016-3c1f-4d67-b909-c1635b5f2892 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:35.650717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc260016-3c1f-4d67-b909-c1635b5f2892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:35.652140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc260016-3c1f-4d67-b909-c1635b5f2892 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:35.653664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:35.663587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 138/282] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:35.670000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4f582-ae80-4977-a567-602c983a9c68 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:35.836941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4f582-ae80-4977-a567-602c983a9c68 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:35.879615 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35d941-0f8b-4a69-b7a7-c4bbad8a5b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:35.880453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35d941-0f8b-4a69-b7a7-c4bbad8a5b86 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:35.880900 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35d941-0f8b-4a69-b7a7-c4bbad8a5b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:35.881314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35d941-0f8b-4a69-b7a7-c4bbad8a5b86 None admin] PATH_INFO: `/v3/users/44d73cd242f14b9eb4585201908dc63a` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:35.882732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35d941-0f8b-4a69-b7a7-c4bbad8a5b86 None admin] RBAC: Authorizing `identity:get_user(user_id=44d73cd242f14b9eb4585201908dc63a)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:35.883658 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc260016-3c1f-4d67-b909-c1635b5f2892 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:35.884533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35d941-0f8b-4a69-b7a7-c4bbad8a5b86 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:35.886274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 145/283] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:53:35 2022] GET /identity/v3/users/44d73cd242f14b9eb4585201908dc63a => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:35.893624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35d941-0f8b-4a69-b7a7-c4bbad8a5b86 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:35.896895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:35.898200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc260016-3c1f-4d67-b909-c1635b5f2892 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:35.913798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310e897-8bbc-4dd2-84fe-b363a18aec60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:35.915830 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310e897-8bbc-4dd2-84fe-b363a18aec60 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:35.916744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310e897-8bbc-4dd2-84fe-b363a18aec60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:35.917821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310e897-8bbc-4dd2-84fe-b363a18aec60 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:35.941647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310e897-8bbc-4dd2-84fe-b363a18aec60 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-7310e897-8bbc-4dd2-84fe-b363a18aec60 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:35.953274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:35.954359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:35.955360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:35.958658 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 139/284] 10.222.0.58 () {56 vars in 1135 bytes} [Fri Oct 28 18:53:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:35.960077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17235e22-1471-429b-870b-dffee94a265d None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:35.960495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17235e22-1471-429b-870b-dffee94a265d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:35.960950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17235e22-1471-429b-870b-dffee94a265d None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:35.962341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 140/285] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:35.967564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17235e22-1471-429b-870b-dffee94a265d None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:35.992053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17235e22-1471-429b-870b-dffee94a265d None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:36.001286 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b1d07-f1b6-4a77-ad59-7ab28bc45910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:36.003145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b1d07-f1b6-4a77-ad59-7ab28bc45910 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.003584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b1d07-f1b6-4a77-ad59-7ab28bc45910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.004576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b1d07-f1b6-4a77-ad59-7ab28bc45910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.006958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b1d07-f1b6-4a77-ad59-7ab28bc45910 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:36.007427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b1d07-f1b6-4a77-ad59-7ab28bc45910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:36.010426 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b1d07-f1b6-4a77-ad59-7ab28bc45910 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:36.028938 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 141/286] 10.222.0.58 () {56 vars in 1132 bytes} [Fri Oct 28 18:53:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:36.030586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f671e-6d2e-4c31-9d3c-00f534c56a4f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.033975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f671e-6d2e-4c31-9d3c-00f534c56a4f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.036422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f671e-6d2e-4c31-9d3c-00f534c56a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.076880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-7e9f671e-6d2e-4c31-9d3c-00f534c56a4f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:36.079409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7e9f671e-6d2e-4c31-9d3c-00f534c56a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:36.086653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35d941-0f8b-4a69-b7a7-c4bbad8a5b86 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:36.098680 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 142/287] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 28 18:53:36.099940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f671e-6d2e-4c31-9d3c-00f534c56a4f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:36.116085 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f671e-6d2e-4c31-9d3c-00f534c56a4f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:36.126508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8162ef1-29d3-4f6e-a7b3-6e1b62ac8ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:36.127453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8162ef1-29d3-4f6e-a7b3-6e1b62ac8ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.127941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8162ef1-29d3-4f6e-a7b3-6e1b62ac8ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.128380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8162ef1-29d3-4f6e-a7b3-6e1b62ac8ca5 admin admin] PATH_INFO: `/v3/groups/9a1e5f068a32416cbe47d515075df556` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.129707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8162ef1-29d3-4f6e-a7b3-6e1b62ac8ca5 admin admin] RBAC: Authorizing `identity:get_group(group_id=9a1e5f068a32416cbe47d515075df556)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:36.131972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8162ef1-29d3-4f6e-a7b3-6e1b62ac8ca5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:36.133727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 143/288] 10.222.0.58 () {56 vars in 1201 bytes} [Fri Oct 28 18:53:36 2022] GET /identity/v3/groups/9a1e5f068a32416cbe47d515075df556 => generated 226 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:36.140314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8162ef1-29d3-4f6e-a7b3-6e1b62ac8ca5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:36.148224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a2433-2762-4154-9eaa-722eb942eabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:36.148851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a2433-2762-4154-9eaa-722eb942eabf None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.149132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a2433-2762-4154-9eaa-722eb942eabf None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.155023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a2433-2762-4154-9eaa-722eb942eabf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.155023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a2433-2762-4154-9eaa-722eb942eabf None admin] RBAC: Adding query filter params (role.id=5981233f19954c69b900f0bbdbb84fd0, user.id=44d73cd242f14b9eb4585201908dc63a, scope.system=all) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:36.155023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a2433-2762-4154-9eaa-722eb942eabf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:36.155023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a2433-2762-4154-9eaa-722eb942eabf None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:36.158373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8162ef1-29d3-4f6e-a7b3-6e1b62ac8ca5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:36.167064 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41754986-3672-4941-acd1-50e17f1e5b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:36.167939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41754986-3672-4941-acd1-50e17f1e5b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.168354 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41754986-3672-4941-acd1-50e17f1e5b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.168808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41754986-3672-4941-acd1-50e17f1e5b4f admin admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.170673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41754986-3672-4941-acd1-50e17f1e5b4f admin admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:36.172993 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41754986-3672-4941-acd1-50e17f1e5b4f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:36.174879 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 144/289] 10.222.0.58 () {56 vars in 1207 bytes} [Fri Oct 28 18:53:36 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:36.180653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41754986-3672-4941-acd1-50e17f1e5b4f admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:36.183167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 146/290] 10.222.0.58 () {56 vars in 1308 bytes} [Fri Oct 28 18:53:35 2022] GET /identity/v3/role_assignments?user.id=44d73cd242f14b9eb4585201908dc63a&scope.system=all&role.id=5981233f19954c69b900f0bbdbb84fd0 => generated 510 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:36.196687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41754986-3672-4941-acd1-50e17f1e5b4f admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:36.207613 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242c8a8-0d63-4876-87e1-a3a535120b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:36.208585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242c8a8-0d63-4876-87e1-a3a535120b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.209014 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242c8a8-0d63-4876-87e1-a3a535120b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.209458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242c8a8-0d63-4876-87e1-a3a535120b84 admin admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/groups/9a1e5f068a32416cbe47d515075df556/roles/0481df198a1a42f0a00358e0b9a938c5` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.211856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242c8a8-0d63-4876-87e1-a3a535120b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccdf5ff122f47538ebe72016c0cab36, group_id=9a1e5f068a32416cbe47d515075df556, role_id=0481df198a1a42f0a00358e0b9a938c5)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:36.214778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242c8a8-0d63-4876-87e1-a3a535120b84 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:36.242276 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 145/291] 10.222.0.58 () {58 vars in 1463 bytes} [Fri Oct 28 18:53:36 2022] PUT /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36/groups/9a1e5f068a32416cbe47d515075df556/roles/0481df198a1a42f0a00358e0b9a938c5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 18:53:36.676730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a2433-2762-4154-9eaa-722eb942eabf None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:36.719459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4271a7-2a49-4c1e-83a6-ac7a3c35f57c None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.719954 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4271a7-2a49-4c1e-83a6-ac7a3c35f57c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.720054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4271a7-2a49-4c1e-83a6-ac7a3c35f57c None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.721326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 146/292] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:36.729746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ee903-ae3c-4db9-b234-efab0bd21fb0 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.729746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ee903-ae3c-4db9-b234-efab0bd21fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.729746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ee903-ae3c-4db9-b234-efab0bd21fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.761041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-0b2ee903-ae3c-4db9-b234-efab0bd21fb0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:36.763866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0b2ee903-ae3c-4db9-b234-efab0bd21fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:36.792191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a2433-2762-4154-9eaa-722eb942eabf None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:36.801428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74683458-76a2-4612-bacc-83de38fc7090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:36.803870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74683458-76a2-4612-bacc-83de38fc7090 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.804353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74683458-76a2-4612-bacc-83de38fc7090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.804837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74683458-76a2-4612-bacc-83de38fc7090 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.823324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:36.831751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74683458-76a2-4612-bacc-83de38fc7090 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-74683458-76a2-4612-bacc-83de38fc7090 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:36.844999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:36.845922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:36.846708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:36.847973 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 147/293] 10.222.0.58 () {56 vars in 1117 bytes} [Fri Oct 28 18:53:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:36.853896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74683458-76a2-4612-bacc-83de38fc7090 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:36.869045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74683458-76a2-4612-bacc-83de38fc7090 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:36.883668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1a0fe-9f43-4bb7-b9c1-b14bf48abb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:36.884671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1a0fe-9f43-4bb7-b9c1-b14bf48abb18 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.885157 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1a0fe-9f43-4bb7-b9c1-b14bf48abb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.885657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1a0fe-9f43-4bb7-b9c1-b14bf48abb18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.886528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1a0fe-9f43-4bb7-b9c1-b14bf48abb18 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:36.887106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1a0fe-9f43-4bb7-b9c1-b14bf48abb18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:36.888994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1a0fe-9f43-4bb7-b9c1-b14bf48abb18 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:36.893010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 147/294] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:36.905564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 148/295] 10.222.0.58 () {56 vars in 1120 bytes} [Fri Oct 28 18:53:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:36.916107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ee903-ae3c-4db9-b234-efab0bd21fb0 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:36.938469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ee903-ae3c-4db9-b234-efab0bd21fb0 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:36.951647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dba49-9442-4686-9f25-9c8898fcfcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:36.952311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dba49-9442-4686-9f25-9c8898fcfcda admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.952589 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dba49-9442-4686-9f25-9c8898fcfcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.952861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dba49-9442-4686-9f25-9c8898fcfcda admin admin] PATH_INFO: `/v3/groups/0da63fc8bdb14ef0a216f1a0221879ba` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.958330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8dba49-9442-4686-9f25-9c8898fcfcda admin admin] RBAC: Authorizing `identity:get_group(group_id=0da63fc8bdb14ef0a216f1a0221879ba)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:36.960268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8dba49-9442-4686-9f25-9c8898fcfcda admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:36.962013 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 148/296] 10.222.0.58 () {56 vars in 1201 bytes} [Fri Oct 28 18:53:36 2022] GET /identity/v3/groups/0da63fc8bdb14ef0a216f1a0221879ba => generated 229 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:36.967714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1a0fe-9f43-4bb7-b9c1-b14bf48abb18 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:36.984938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1a0fe-9f43-4bb7-b9c1-b14bf48abb18 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:36.989751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006953b-890c-4356-b423-c9f99576238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:36.990628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006953b-890c-4356-b423-c9f99576238f admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:36.991150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006953b-890c-4356-b423-c9f99576238f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:36.991618 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006953b-890c-4356-b423-c9f99576238f admin admin] PATH_INFO: `/v3/projects/b28e4680a856417f99f9b94c78ac743d` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:36.992917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006953b-890c-4356-b423-c9f99576238f admin admin] RBAC: Authorizing `identity:get_project(project_id=b28e4680a856417f99f9b94c78ac743d)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:36.994910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006953b-890c-4356-b423-c9f99576238f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:36.996767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 149/297] 10.222.0.58 () {56 vars in 1207 bytes} [Fri Oct 28 18:53:36 2022] GET /identity/v3/projects/b28e4680a856417f99f9b94c78ac743d => generated 344 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:37.003109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dba49-9442-4686-9f25-9c8898fcfcda admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:37.016075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dba49-9442-4686-9f25-9c8898fcfcda admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:37.023066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4267e0c-fc80-4ca9-be87-cb3700692a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:37.023725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4267e0c-fc80-4ca9-be87-cb3700692a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:37.024000 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4267e0c-fc80-4ca9-be87-cb3700692a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:37.024268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4267e0c-fc80-4ca9-be87-cb3700692a41 admin admin] PATH_INFO: `/v3/projects/b28e4680a856417f99f9b94c78ac743d/groups/0da63fc8bdb14ef0a216f1a0221879ba/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:37.032295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4267e0c-fc80-4ca9-be87-cb3700692a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28e4680a856417f99f9b94c78ac743d, group_id=0da63fc8bdb14ef0a216f1a0221879ba, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:37.034997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4267e0c-fc80-4ca9-be87-cb3700692a41 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:37.063506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 149/298] 10.222.0.58 () {58 vars in 1463 bytes} [Fri Oct 28 18:53:37 2022] PUT /identity/v3/projects/b28e4680a856417f99f9b94c78ac743d/groups/0da63fc8bdb14ef0a216f1a0221879ba/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:37.242807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006953b-890c-4356-b423-c9f99576238f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:37.388232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006953b-890c-4356-b423-c9f99576238f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:37.431622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab49d52-1858-40ad-92fc-97a69b5254dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:37.432285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab49d52-1858-40ad-92fc-97a69b5254dd None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:37.432566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab49d52-1858-40ad-92fc-97a69b5254dd None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:37.432838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab49d52-1858-40ad-92fc-97a69b5254dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:37.453578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab49d52-1858-40ad-92fc-97a69b5254dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-4ab49d52-1858-40ad-92fc-97a69b5254dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:37.459242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:37.460176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:37.460941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 150/299] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:37.468213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4267e0c-fc80-4ca9-be87-cb3700692a41 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:37.552038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:37.604979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4267e0c-fc80-4ca9-be87-cb3700692a41 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:37.650007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611644e5-f0f2-4fc2-bc64-baf0d026d838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:37.650625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611644e5-f0f2-4fc2-bc64-baf0d026d838 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:37.650901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611644e5-f0f2-4fc2-bc64-baf0d026d838 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:37.651197 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611644e5-f0f2-4fc2-bc64-baf0d026d838 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:37.651860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611644e5-f0f2-4fc2-bc64-baf0d026d838 None admin] RBAC: Adding query filter params (name=member) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:37.652156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611644e5-f0f2-4fc2-bc64-baf0d026d838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:37.653533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611644e5-f0f2-4fc2-bc64-baf0d026d838 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:37.665969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 150/300] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:37.671415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab49d52-1858-40ad-92fc-97a69b5254dd None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:37.737086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:37.742934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11e086-5889-4619-8a0f-8fdac8fbdb36 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:37.743402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11e086-5889-4619-8a0f-8fdac8fbdb36 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:37.743503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11e086-5889-4619-8a0f-8fdac8fbdb36 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:37.745107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 151/301] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:37.752327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29202889-e489-4b9d-a62f-66d627749c16 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:37.752327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29202889-e489-4b9d-a62f-66d627749c16 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:37.752327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29202889-e489-4b9d-a62f-66d627749c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:37.786558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-29202889-e489-4b9d-a62f-66d627749c16 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:37.787769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-29202889-e489-4b9d-a62f-66d627749c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:37.843704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab49d52-1858-40ad-92fc-97a69b5254dd None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:37.894715 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2f5cb-077a-4668-94cb-99be037a8912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:37.895801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2f5cb-077a-4668-94cb-99be037a8912 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:37.896527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2f5cb-077a-4668-94cb-99be037a8912 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:37.896998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2f5cb-077a-4668-94cb-99be037a8912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:37.939788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 152/302] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:37.974770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2f5cb-077a-4668-94cb-99be037a8912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:37.976665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2f5cb-077a-4668-94cb-99be037a8912 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:38.031305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:38.040246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21851036-429c-4fa3-8ce1-02c4be80029f None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:38.040533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21851036-429c-4fa3-8ce1-02c4be80029f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:38.040808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21851036-429c-4fa3-8ce1-02c4be80029f None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:38.047119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 153/303] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:38.048381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4f002-5d5e-41cd-bd6f-9b40331faaab None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:38.048662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4f002-5d5e-41cd-bd6f-9b40331faaab None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:38.049035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4f002-5d5e-41cd-bd6f-9b40331faaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:38.080818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-23f4f002-5d5e-41cd-bd6f-9b40331faaab None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:38.082068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-23f4f002-5d5e-41cd-bd6f-9b40331faaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:38.116252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 151/304] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 28 18:53:38.121996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2f5cb-077a-4668-94cb-99be037a8912 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:38.164976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:38.208026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 154/305] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:38.266749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2f5cb-077a-4668-94cb-99be037a8912 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:38.278071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4f002-5d5e-41cd-bd6f-9b40331faaab None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:38.337339 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3650f-28a5-48a8-9085-1578a2dc7929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:38.337971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3650f-28a5-48a8-9085-1578a2dc7929 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:38.338248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3650f-28a5-48a8-9085-1578a2dc7929 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:38.338513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3650f-28a5-48a8-9085-1578a2dc7929 None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:38.343421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3650f-28a5-48a8-9085-1578a2dc7929 None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:38.344873 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3650f-28a5-48a8-9085-1578a2dc7929 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:38.348780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 152/306] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:38 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:38.352597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3650f-28a5-48a8-9085-1578a2dc7929 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:38.380139 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:38.458206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4f002-5d5e-41cd-bd6f-9b40331faaab None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:38.522506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3650f-28a5-48a8-9085-1578a2dc7929 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:38.525905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0c56e-fee9-4351-a310-93f611c207e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:38.526540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0c56e-fee9-4351-a310-93f611c207e6 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:38.526820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0c56e-fee9-4351-a310-93f611c207e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:38.527115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0c56e-fee9-4351-a310-93f611c207e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:38.552535 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0c56e-fee9-4351-a310-93f611c207e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-8cf0c56e-fee9-4351-a310-93f611c207e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:38.558342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:38.559236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:38.560065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 155/307] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:38.566969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0c56e-fee9-4351-a310-93f611c207e6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:38.600550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa8cfc-99b4-4b2b-9c57-d3558b9bc0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:38.601202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa8cfc-99b4-4b2b-9c57-d3558b9bc0ce None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:38.601482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa8cfc-99b4-4b2b-9c57-d3558b9bc0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:38.601765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa8cfc-99b4-4b2b-9c57-d3558b9bc0ce None admin] PATH_INFO: `/v3/groups/9a1e5f068a32416cbe47d515075df556` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:38.604200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa8cfc-99b4-4b2b-9c57-d3558b9bc0ce None admin] RBAC: Authorizing `identity:get_group(group_id=9a1e5f068a32416cbe47d515075df556)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:38.604666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa8cfc-99b4-4b2b-9c57-d3558b9bc0ce None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:38.606380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 153/308] 10.222.0.58 () {56 vars in 1180 bytes} [Fri Oct 28 18:53:38 2022] GET /identity/v3/groups/9a1e5f068a32416cbe47d515075df556 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:38.607268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa8cfc-99b4-4b2b-9c57-d3558b9bc0ce None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:38.642097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:38.717171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:38.733845 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0c56e-fee9-4351-a310-93f611c207e6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:38.741417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa8cfc-99b4-4b2b-9c57-d3558b9bc0ce None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:38.783853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe04f45-008c-47e4-8f7c-d2de178cc7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:38.784487 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe04f45-008c-47e4-8f7c-d2de178cc7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:38.784767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe04f45-008c-47e4-8f7c-d2de178cc7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:38.785037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe04f45-008c-47e4-8f7c-d2de178cc7a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:38.788216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccc573-67cd-48e5-bc8a-18f0eb44fa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:38.789387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccc573-67cd-48e5-bc8a-18f0eb44fa8d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:38.789994 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccc573-67cd-48e5-bc8a-18f0eb44fa8d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:38.790585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccc573-67cd-48e5-bc8a-18f0eb44fa8d None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:38.791707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccc573-67cd-48e5-bc8a-18f0eb44fa8d None admin] RBAC: Adding query filter params (name=member) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:38.792476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccc573-67cd-48e5-bc8a-18f0eb44fa8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:38.794601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccc573-67cd-48e5-bc8a-18f0eb44fa8d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:38.795842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe04f45-008c-47e4-8f7c-d2de178cc7a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-cbe04f45-008c-47e4-8f7c-d2de178cc7a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:38.802711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:38.803726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:38.804509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 154/309] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:53:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:38.805149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe04f45-008c-47e4-8f7c-d2de178cc7a0 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:38.808470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 156/310] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:53:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:38.811153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccc573-67cd-48e5-bc8a-18f0eb44fa8d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:38.928473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccc573-67cd-48e5-bc8a-18f0eb44fa8d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:38.930008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:38.933569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe04f45-008c-47e4-8f7c-d2de178cc7a0 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:38.963485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7281f-a538-43c4-9580-da0a0f9576b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:38.964101 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7281f-a538-43c4-9580-da0a0f9576b5 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:38.964378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7281f-a538-43c4-9580-da0a0f9576b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:38.964645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7281f-a538-43c4-9580-da0a0f9576b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:38.965419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7281f-a538-43c4-9580-da0a0f9576b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:38.965600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7281f-a538-43c4-9580-da0a0f9576b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:38.967546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7281f-a538-43c4-9580-da0a0f9576b5 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:38.974905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427f2df-f549-4d79-b062-4af990fd2637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:38.975808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427f2df-f549-4d79-b062-4af990fd2637 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:38.976332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 157/311] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:38.976439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427f2df-f549-4d79-b062-4af990fd2637 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:38.976937 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427f2df-f549-4d79-b062-4af990fd2637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:38.977152 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7281f-a538-43c4-9580-da0a0f9576b5 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:38.978172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427f2df-f549-4d79-b062-4af990fd2637 None admin] RBAC: Adding query filter params (group.id=9a1e5f068a32416cbe47d515075df556, role.id=3c56cfa91fba453da353f21d7d6c1189, scope.project.id=96c771455a484eb080ab85c17582f652) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:38.978677 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427f2df-f549-4d79-b062-4af990fd2637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:38.980368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427f2df-f549-4d79-b062-4af990fd2637 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:38.988258 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 155/312] 10.222.0.58 () {56 vars in 1376 bytes} [Fri Oct 28 18:53:38 2022] GET /identity/v3/role_assignments?group.id=9a1e5f068a32416cbe47d515075df556&scope.project.id=96c771455a484eb080ab85c17582f652&role.id=3c56cfa91fba453da353f21d7d6c1189 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:38.989380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427f2df-f549-4d79-b062-4af990fd2637 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:38.999417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:39.093218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7281f-a538-43c4-9580-da0a0f9576b5 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:39.117479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427f2df-f549-4d79-b062-4af990fd2637 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:39.130093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a74eca-fc12-4a6b-b978-1b6b08151997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:39.130714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a74eca-fc12-4a6b-b978-1b6b08151997 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:39.131013 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a74eca-fc12-4a6b-b978-1b6b08151997 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:39.131335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a74eca-fc12-4a6b-b978-1b6b08151997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:39.158801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ce4ff-4459-4bac-b6e5-a544d9997604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:39.159904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ce4ff-4459-4bac-b6e5-a544d9997604 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:39.160383 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ce4ff-4459-4bac-b6e5-a544d9997604 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:39.160899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ce4ff-4459-4bac-b6e5-a544d9997604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:39.170247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:39.213520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a74eca-fc12-4a6b-b978-1b6b08151997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:39.214963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a74eca-fc12-4a6b-b978-1b6b08151997 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:39.240441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ce4ff-4459-4bac-b6e5-a544d9997604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:39.242307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ce4ff-4459-4bac-b6e5-a544d9997604 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:39.250830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:39.326734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 158/313] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:39.332454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a74eca-fc12-4a6b-b978-1b6b08151997 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:39.359769 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 156/314] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:39.367295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ce4ff-4459-4bac-b6e5-a544d9997604 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:39.407561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:39.456754 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a74eca-fc12-4a6b-b978-1b6b08151997 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:39.499370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ce4ff-4459-4bac-b6e5-a544d9997604 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:39.504674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bb453-8797-40b1-b9fc-b51dee9322a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:39.505318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb453-8797-40b1-b9fc-b51dee9322a8 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:39.505596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb453-8797-40b1-b9fc-b51dee9322a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:39.505871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb453-8797-40b1-b9fc-b51dee9322a8 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:39.507108 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bb453-8797-40b1-b9fc-b51dee9322a8 None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:39.508276 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:39.508560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bb453-8797-40b1-b9fc-b51dee9322a8 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:39.510404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 159/315] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:39 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:39.511372 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e18af4-0e63-4e94-a0e9-46dc1d098c37 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:39.511653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e18af4-0e63-4e94-a0e9-46dc1d098c37 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:39.511928 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e18af4-0e63-4e94-a0e9-46dc1d098c37 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:39.513966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 160/316] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:39.519016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e18af4-0e63-4e94-a0e9-46dc1d098c37 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:39.546667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccf35d-76e2-4b18-8769-845cbff61837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:39.547326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccf35d-76e2-4b18-8769-845cbff61837 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:39.547603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccf35d-76e2-4b18-8769-845cbff61837 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:39.547876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccf35d-76e2-4b18-8769-845cbff61837 None admin] PATH_INFO: `/v3/projects/96c771455a484eb080ab85c17582f652` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:39.550115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccf35d-76e2-4b18-8769-845cbff61837 None admin] RBAC: Authorizing `identity:get_project(project_id=96c771455a484eb080ab85c17582f652)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:39.551630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccf35d-76e2-4b18-8769-845cbff61837 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:39.553569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 157/317] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:39 2022] GET /identity/v3/projects/96c771455a484eb080ab85c17582f652 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:39.554706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09f15e-b06e-416e-9e37-379b195a6521 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:39.555035 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09f15e-b06e-416e-9e37-379b195a6521 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:39.559381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09f15e-b06e-416e-9e37-379b195a6521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:39.583796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2a09f15e-b06e-416e-9e37-379b195a6521 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:39.583796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2a09f15e-b06e-416e-9e37-379b195a6521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:39.653796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e18af4-0e63-4e94-a0e9-46dc1d098c37 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:39.674158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 158/318] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:39.675096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09f15e-b06e-416e-9e37-379b195a6521 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:39.692499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:39.694110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4a6e-d16e-423a-aac2-8b2e6742471e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:39.694611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4a6e-d16e-423a-aac2-8b2e6742471e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:39.695075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4a6e-d16e-423a-aac2-8b2e6742471e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:39.695580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4a6e-d16e-423a-aac2-8b2e6742471e None admin] PATH_INFO: `/v3/groups/9a1e5f068a32416cbe47d515075df556` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:39.703992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca4a6e-d16e-423a-aac2-8b2e6742471e None admin] RBAC: Authorizing `identity:get_group(group_id=9a1e5f068a32416cbe47d515075df556)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:39.705458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca4a6e-d16e-423a-aac2-8b2e6742471e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:39.709683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 161/319] 10.222.0.58 () {56 vars in 1180 bytes} [Fri Oct 28 18:53:39 2022] GET /identity/v3/groups/9a1e5f068a32416cbe47d515075df556 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:39.714767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4a6e-d16e-423a-aac2-8b2e6742471e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:39.808965 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09f15e-b06e-416e-9e37-379b195a6521 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:39.814054 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:39.857755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4a6e-d16e-423a-aac2-8b2e6742471e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:39.859557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f9086-fe81-4514-9088-dc99fce3934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:39.860304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f9086-fe81-4514-9088-dc99fce3934b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:39.860587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f9086-fe81-4514-9088-dc99fce3934b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:39.860858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f9086-fe81-4514-9088-dc99fce3934b None admin] PATH_INFO: `/v3/groups/9a1e5f068a32416cbe47d515075df556` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:39.861973 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f9086-fe81-4514-9088-dc99fce3934b None admin] RBAC: Authorizing `identity:get_group(group_id=9a1e5f068a32416cbe47d515075df556)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:39.863447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f9086-fe81-4514-9088-dc99fce3934b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:39.867104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 159/320] 10.222.0.58 () {56 vars in 1180 bytes} [Fri Oct 28 18:53:39 2022] GET /identity/v3/groups/9a1e5f068a32416cbe47d515075df556 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:39.874080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f9086-fe81-4514-9088-dc99fce3934b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:39.899551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92df9f-447d-485c-9981-d333c6cfac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:39.900161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92df9f-447d-485c-9981-d333c6cfac98 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:39.900438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92df9f-447d-485c-9981-d333c6cfac98 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:39.900709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92df9f-447d-485c-9981-d333c6cfac98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:39.901434 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92df9f-447d-485c-9981-d333c6cfac98 None admin] RBAC: Adding query filter params (group.id=9a1e5f068a32416cbe47d515075df556, role.id=3c56cfa91fba453da353f21d7d6c1189, scope.project.id=2ccdf5ff122f47538ebe72016c0cab36) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:39.901739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92df9f-447d-485c-9981-d333c6cfac98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:39.903194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92df9f-447d-485c-9981-d333c6cfac98 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:39.909463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 162/321] 10.222.0.58 () {56 vars in 1376 bytes} [Fri Oct 28 18:53:39 2022] GET /identity/v3/role_assignments?group.id=9a1e5f068a32416cbe47d515075df556&scope.project.id=2ccdf5ff122f47538ebe72016c0cab36&role.id=3c56cfa91fba453da353f21d7d6c1189 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:39.973389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92df9f-447d-485c-9981-d333c6cfac98 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:39.999369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f9086-fe81-4514-9088-dc99fce3934b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:40.050955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3054035-49f1-41aa-8554-379a0b869d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:40.051628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3054035-49f1-41aa-8554-379a0b869d69 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:40.051907 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3054035-49f1-41aa-8554-379a0b869d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:40.053995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3054035-49f1-41aa-8554-379a0b869d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:40.053995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3054035-49f1-41aa-8554-379a0b869d69 None admin] RBAC: Adding query filter params (group.id=9a1e5f068a32416cbe47d515075df556, role.id=0481df198a1a42f0a00358e0b9a938c5, scope.project.id=96c771455a484eb080ab85c17582f652) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:40.053995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3054035-49f1-41aa-8554-379a0b869d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:40.055195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3054035-49f1-41aa-8554-379a0b869d69 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:40.064391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 160/322] 10.222.0.58 () {56 vars in 1376 bytes} [Fri Oct 28 18:53:39 2022] GET /identity/v3/role_assignments?group.id=9a1e5f068a32416cbe47d515075df556&scope.project.id=96c771455a484eb080ab85c17582f652&role.id=0481df198a1a42f0a00358e0b9a938c5 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:40.091191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:40.145859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92df9f-447d-485c-9981-d333c6cfac98 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:40.154572 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465eaed-40ef-42ac-951d-a9c344bf51f1 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:40.154854 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465eaed-40ef-42ac-951d-a9c344bf51f1 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:40.155259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465eaed-40ef-42ac-951d-a9c344bf51f1 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:40.156671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 161/323] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:40.163522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c49116-da2e-4ccc-9eb5-a3b184a7c88e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:40.164121 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c49116-da2e-4ccc-9eb5-a3b184a7c88e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:40.164594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c49116-da2e-4ccc-9eb5-a3b184a7c88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:40.193479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-e9c49116-da2e-4ccc-9eb5-a3b184a7c88e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:40.193479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e9c49116-da2e-4ccc-9eb5-a3b184a7c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:40.196163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e239ea-9457-452c-8380-f73a52025030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:40.198145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e239ea-9457-452c-8380-f73a52025030 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:40.198697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e239ea-9457-452c-8380-f73a52025030 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:40.199226 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e239ea-9457-452c-8380-f73a52025030 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:40.218786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e239ea-9457-452c-8380-f73a52025030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-c2e239ea-9457-452c-8380-f73a52025030 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:40.221441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:40.222285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:40.223604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 163/324] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:53:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:40.232964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e239ea-9457-452c-8380-f73a52025030 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:40.238201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:40.292186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 162/325] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:40.353296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e239ea-9457-452c-8380-f73a52025030 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:40.379669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:40.394493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54573e20-9aa6-4a19-9d4d-6ac703f8e3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:40.395549 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54573e20-9aa6-4a19-9d4d-6ac703f8e3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:40.396086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54573e20-9aa6-4a19-9d4d-6ac703f8e3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:40.396547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54573e20-9aa6-4a19-9d4d-6ac703f8e3e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:40.397398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54573e20-9aa6-4a19-9d4d-6ac703f8e3e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:40.397889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54573e20-9aa6-4a19-9d4d-6ac703f8e3e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:40.399561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54573e20-9aa6-4a19-9d4d-6ac703f8e3e0 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:40.407987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 164/326] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:40.413720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c49116-da2e-4ccc-9eb5-a3b184a7c88e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:40.517294 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c49116-da2e-4ccc-9eb5-a3b184a7c88e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:40.551877 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a6886-82ea-41b6-8b86-fba673e3f661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:40.552704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a6886-82ea-41b6-8b86-fba673e3f661 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:40.553153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a6886-82ea-41b6-8b86-fba673e3f661 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:40.553418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a6886-82ea-41b6-8b86-fba673e3f661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:40.561723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54573e20-9aa6-4a19-9d4d-6ac703f8e3e0 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:40.589547 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:40.627630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a6886-82ea-41b6-8b86-fba673e3f661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:40.629004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a6886-82ea-41b6-8b86-fba673e3f661 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:40.680319 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54573e20-9aa6-4a19-9d4d-6ac703f8e3e0 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:40.721741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc1202-d64b-4837-9244-5a4924979c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:40.722379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc1202-d64b-4837-9244-5a4924979c33 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:40.722657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc1202-d64b-4837-9244-5a4924979c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:40.722929 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc1202-d64b-4837-9244-5a4924979c33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:40.736857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc1202-d64b-4837-9244-5a4924979c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-2ddc1202-d64b-4837-9244-5a4924979c33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:40.742339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:40.743231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:40.744023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 165/327] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:53:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:40.744675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 163/328] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:40.751413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc1202-d64b-4837-9244-5a4924979c33 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:40.752022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a6886-82ea-41b6-8b86-fba673e3f661 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:40.803978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:40.823618 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:40.864481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a6886-82ea-41b6-8b86-fba673e3f661 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:40.867948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc1202-d64b-4837-9244-5a4924979c33 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:40.907694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b309c-b3d4-4368-8dbd-674ed2de1ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:40.908114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d56e3f-cdd1-4e9a-bebd-c8bed631b465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:40.908246 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b309c-b3d4-4368-8dbd-674ed2de1ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:40.908914 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b309c-b3d4-4368-8dbd-674ed2de1ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:40.909015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d56e3f-cdd1-4e9a-bebd-c8bed631b465 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:40.909086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b309c-b3d4-4368-8dbd-674ed2de1ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:40.909151 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d56e3f-cdd1-4e9a-bebd-c8bed631b465 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:40.909327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d56e3f-cdd1-4e9a-bebd-c8bed631b465 None admin] PATH_INFO: `/v3/projects/2ccdf5ff122f47538ebe72016c0cab36` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:40.909416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b309c-b3d4-4368-8dbd-674ed2de1ef7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:40.909710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b309c-b3d4-4368-8dbd-674ed2de1ef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:40.910418 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d56e3f-cdd1-4e9a-bebd-c8bed631b465 None admin] RBAC: Authorizing `identity:get_project(project_id=2ccdf5ff122f47538ebe72016c0cab36)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:40.911101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b309c-b3d4-4368-8dbd-674ed2de1ef7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:40.911867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d56e3f-cdd1-4e9a-bebd-c8bed631b465 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:40.913485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 166/329] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:40 2022] GET /identity/v3/projects/2ccdf5ff122f47538ebe72016c0cab36 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:40.918921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d56e3f-cdd1-4e9a-bebd-c8bed631b465 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:40.924785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 164/330] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:40.930735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b309c-b3d4-4368-8dbd-674ed2de1ef7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:41.044023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d56e3f-cdd1-4e9a-bebd-c8bed631b465 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:41.048008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b309c-b3d4-4368-8dbd-674ed2de1ef7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:41.061116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:41.076915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:41.085480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201b61f-9204-4cf2-9609-be7bd02c00cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:41.086116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201b61f-9204-4cf2-9609-be7bd02c00cf None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:41.086393 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201b61f-9204-4cf2-9609-be7bd02c00cf None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:41.086660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201b61f-9204-4cf2-9609-be7bd02c00cf None admin] PATH_INFO: `/v3/groups/9a1e5f068a32416cbe47d515075df556` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:41.087635 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e6d74-c064-4ddb-a782-fc639f0a01b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:41.087784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201b61f-9204-4cf2-9609-be7bd02c00cf None admin] RBAC: Authorizing `identity:get_group(group_id=9a1e5f068a32416cbe47d515075df556)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:41.088242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e6d74-c064-4ddb-a782-fc639f0a01b0 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:41.088525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e6d74-c064-4ddb-a782-fc639f0a01b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:41.088801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e6d74-c064-4ddb-a782-fc639f0a01b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:41.089183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201b61f-9204-4cf2-9609-be7bd02c00cf None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:41.095223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 167/331] 10.222.0.58 () {56 vars in 1180 bytes} [Fri Oct 28 18:53:40 2022] GET /identity/v3/groups/9a1e5f068a32416cbe47d515075df556 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:53:41.097030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201b61f-9204-4cf2-9609-be7bd02c00cf None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:41.172583 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e6d74-c064-4ddb-a782-fc639f0a01b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:41.173986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e6d74-c064-4ddb-a782-fc639f0a01b0 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:41.222657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201b61f-9204-4cf2-9609-be7bd02c00cf None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:41.254863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381801d-f32e-41f7-81c4-8b596fcf3f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:41.255500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381801d-f32e-41f7-81c4-8b596fcf3f88 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:41.255776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381801d-f32e-41f7-81c4-8b596fcf3f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:41.256044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381801d-f32e-41f7-81c4-8b596fcf3f88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:41.256750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381801d-f32e-41f7-81c4-8b596fcf3f88 None admin] RBAC: Adding query filter params (group.id=9a1e5f068a32416cbe47d515075df556, role.id=0481df198a1a42f0a00358e0b9a938c5, scope.project.id=2ccdf5ff122f47538ebe72016c0cab36) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:41.257053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381801d-f32e-41f7-81c4-8b596fcf3f88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:41.258465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381801d-f32e-41f7-81c4-8b596fcf3f88 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:41.263953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 168/332] 10.222.0.58 () {56 vars in 1376 bytes} [Fri Oct 28 18:53:41 2022] GET /identity/v3/role_assignments?group.id=9a1e5f068a32416cbe47d515075df556&scope.project.id=2ccdf5ff122f47538ebe72016c0cab36&role.id=0481df198a1a42f0a00358e0b9a938c5 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:41.279298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 165/333] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:41.285135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381801d-f32e-41f7-81c4-8b596fcf3f88 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:41.334825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:41.391201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381801d-f32e-41f7-81c4-8b596fcf3f88 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:41.425724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440c0ee-4435-46e7-8778-d16801f37400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:41.426275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440c0ee-4435-46e7-8778-d16801f37400 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:41.426552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440c0ee-4435-46e7-8778-d16801f37400 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:41.426822 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440c0ee-4435-46e7-8778-d16801f37400 None admin] PATH_INFO: `/v3/projects/b28e4680a856417f99f9b94c78ac743d` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:41.427968 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440c0ee-4435-46e7-8778-d16801f37400 None admin] RBAC: Authorizing `identity:get_project(project_id=b28e4680a856417f99f9b94c78ac743d)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:41.429394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440c0ee-4435-46e7-8778-d16801f37400 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:41.431200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 169/334] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:53:41 2022] GET /identity/v3/projects/b28e4680a856417f99f9b94c78ac743d => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:41.436606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e6d74-c064-4ddb-a782-fc639f0a01b0 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:41.464532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:41.538760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e6d74-c064-4ddb-a782-fc639f0a01b0 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:41.572202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90df5fb-8582-4fd6-b1ca-8d0fb33a8f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:41.572831 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90df5fb-8582-4fd6-b1ca-8d0fb33a8f7c None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:41.573115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90df5fb-8582-4fd6-b1ca-8d0fb33a8f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:41.573386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90df5fb-8582-4fd6-b1ca-8d0fb33a8f7c None admin] PATH_INFO: `/v3/groups/0da63fc8bdb14ef0a216f1a0221879ba` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:41.574495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90df5fb-8582-4fd6-b1ca-8d0fb33a8f7c None admin] RBAC: Authorizing `identity:get_group(group_id=0da63fc8bdb14ef0a216f1a0221879ba)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:41.576586 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90df5fb-8582-4fd6-b1ca-8d0fb33a8f7c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:41.578164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 166/335] 10.222.0.58 () {56 vars in 1180 bytes} [Fri Oct 28 18:53:41 2022] GET /identity/v3/groups/0da63fc8bdb14ef0a216f1a0221879ba => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:41.584102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440c0ee-4435-46e7-8778-d16801f37400 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:41.688709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440c0ee-4435-46e7-8778-d16801f37400 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:41.701896 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:41.721745 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6b7b3-e1ca-4de1-9a68-edd9315ee66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:41.722333 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6b7b3-e1ca-4de1-9a68-edd9315ee66a None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:41.722608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6b7b3-e1ca-4de1-9a68-edd9315ee66a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:41.722875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6b7b3-e1ca-4de1-9a68-edd9315ee66a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:41.723619 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6b7b3-e1ca-4de1-9a68-edd9315ee66a None admin] RBAC: Adding query filter params (group.id=0da63fc8bdb14ef0a216f1a0221879ba, role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.project.id=b28e4680a856417f99f9b94c78ac743d) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:41.723923 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6b7b3-e1ca-4de1-9a68-edd9315ee66a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:41.725346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6b7b3-e1ca-4de1-9a68-edd9315ee66a None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:41.731280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 170/336] 10.222.0.58 () {56 vars in 1376 bytes} [Fri Oct 28 18:53:41 2022] GET /identity/v3/role_assignments?group.id=0da63fc8bdb14ef0a216f1a0221879ba&scope.project.id=b28e4680a856417f99f9b94c78ac743d&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:44.250709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d32df-14db-4f6e-b495-0da47cd081dc None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:44.251252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d32df-14db-4f6e-b495-0da47cd081dc None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:44.251252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d32df-14db-4f6e-b495-0da47cd081dc None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:44.252858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 167/337] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:44.268825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b928225-3b09-465b-8ebc-cad91ba53026 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:44.269838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b928225-3b09-465b-8ebc-cad91ba53026 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:44.270301 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b928225-3b09-465b-8ebc-cad91ba53026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:44.296028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6b928225-3b09-465b-8ebc-cad91ba53026 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:44.297240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6b928225-3b09-465b-8ebc-cad91ba53026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:44.389481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 171/338] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:44.498205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839db514-a17f-4f34-a95f-3c65ea11ce19 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:44.498205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839db514-a17f-4f34-a95f-3c65ea11ce19 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:44.498205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839db514-a17f-4f34-a95f-3c65ea11ce19 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:44.498874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 168/339] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:44.504526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da451369-70cb-4401-bc86-e80d18f8d3e8 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:44.504751 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da451369-70cb-4401-bc86-e80d18f8d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:44.505024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da451369-70cb-4401-bc86-e80d18f8d3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:44.533583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-da451369-70cb-4401-bc86-e80d18f8d3e8 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:44.534807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-da451369-70cb-4401-bc86-e80d18f8d3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:44.636485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 172/340] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:44.704555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae42bdc-c07b-47e5-9482-936a84d977bf None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:44.705096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae42bdc-c07b-47e5-9482-936a84d977bf None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:44.705513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae42bdc-c07b-47e5-9482-936a84d977bf None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:44.707027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 169/341] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:44.724644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b94ee3-ae5f-462a-a050-48645d61075c None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:44.725093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b94ee3-ae5f-462a-a050-48645d61075c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:44.725531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b94ee3-ae5f-462a-a050-48645d61075c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:44.748507 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-72b94ee3-ae5f-462a-a050-48645d61075c None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:44.749923 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-72b94ee3-ae5f-462a-a050-48645d61075c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:44.791015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:44.845409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 173/342] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:44.868852 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae42bdc-c07b-47e5-9482-936a84d977bf None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:44.999523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:45.006529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae42bdc-c07b-47e5-9482-936a84d977bf None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:45.011370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b94ee3-ae5f-462a-a050-48645d61075c None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:45.045746 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc01bca-9aa0-4bed-b3d0-d00b61d39718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:45.046283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01bca-9aa0-4bed-b3d0-d00b61d39718 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.046723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01bca-9aa0-4bed-b3d0-d00b61d39718 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.047412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01bca-9aa0-4bed-b3d0-d00b61d39718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.058674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc01bca-9aa0-4bed-b3d0-d00b61d39718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:45.061125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc01bca-9aa0-4bed-b3d0-d00b61d39718 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-0bc01bca-9aa0-4bed-b3d0-d00b61d39718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:45.079948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:45.081066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:45.082086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:45.082086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:45.082086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:45.083946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 170/343] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:45.091552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc01bca-9aa0-4bed-b3d0-d00b61d39718 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:45.206899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b94ee3-ae5f-462a-a050-48645d61075c None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:45.247033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e263-35b5-4197-8374-69a7b90545c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:45.247947 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e263-35b5-4197-8374-69a7b90545c8 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.248366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e263-35b5-4197-8374-69a7b90545c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.248776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e263-35b5-4197-8374-69a7b90545c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.254686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc01bca-9aa0-4bed-b3d0-d00b61d39718 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:45.261224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34e263-35b5-4197-8374-69a7b90545c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:45.263844 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34e263-35b5-4197-8374-69a7b90545c8 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-fa34e263-35b5-4197-8374-69a7b90545c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:45.281512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:45.282862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:45.283760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:45.283760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:45.283760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:45.283760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 174/344] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:45.283760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e263-35b5-4197-8374-69a7b90545c8 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:45.299604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e7751-ebc4-45b2-bdfc-588807e89ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:45.300546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e7751-ebc4-45b2-bdfc-588807e89ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.301080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e7751-ebc4-45b2-bdfc-588807e89ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.301636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e7751-ebc4-45b2-bdfc-588807e89ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.302571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e7751-ebc4-45b2-bdfc-588807e89ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:45.303086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e7751-ebc4-45b2-bdfc-588807e89ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:45.304661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e7751-ebc4-45b2-bdfc-588807e89ea3 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:45.305284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:45.319913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 171/345] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:45.320924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e7751-ebc4-45b2-bdfc-588807e89ea3 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:45.333713 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:45.389716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e263-35b5-4197-8374-69a7b90545c8 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:45.426166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb9606-cc47-4e99-a6e1-2e04663e96ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:45.426792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb9606-cc47-4e99-a6e1-2e04663e96ed None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.427100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb9606-cc47-4e99-a6e1-2e04663e96ed None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.427372 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb9606-cc47-4e99-a6e1-2e04663e96ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.430092 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e7751-ebc4-45b2-bdfc-588807e89ea3 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:45.438692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb9606-cc47-4e99-a6e1-2e04663e96ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:45.440154 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb9606-cc47-4e99-a6e1-2e04663e96ed None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-19bb9606-cc47-4e99-a6e1-2e04663e96ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:45.456265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:45.457350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:45.458295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:45.458295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:45.458295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:45.459657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 175/346] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:45.464205 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80066de6-6ff8-4b60-8a4a-e7255a00b9f5 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.464741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80066de6-6ff8-4b60-8a4a-e7255a00b9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.465020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80066de6-6ff8-4b60-8a4a-e7255a00b9f5 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.467050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 176/347] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:45.467623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80066de6-6ff8-4b60-8a4a-e7255a00b9f5 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:45.472469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c5f8a-934f-4d3a-b6e4-c05059b86cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:45.473086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c5f8a-934f-4d3a-b6e4-c05059b86cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.473365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c5f8a-934f-4d3a-b6e4-c05059b86cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.473638 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c5f8a-934f-4d3a-b6e4-c05059b86cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.474283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c5f8a-934f-4d3a-b6e4-c05059b86cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:45.474586 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c5f8a-934f-4d3a-b6e4-c05059b86cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:45.476006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c5f8a-934f-4d3a-b6e4-c05059b86cb2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:45.487463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 172/348] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:45.488255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c5f8a-934f-4d3a-b6e4-c05059b86cb2 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:45.548582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:45.559237 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:45.586531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80066de6-6ff8-4b60-8a4a-e7255a00b9f5 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:45.596714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c5f8a-934f-4d3a-b6e4-c05059b86cb2 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:45.621457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcfdc1c-0b1e-4766-b0a1-772e9e3eb2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:45.622069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfdc1c-0b1e-4766-b0a1-772e9e3eb2b1 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.622908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfdc1c-0b1e-4766-b0a1-772e9e3eb2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.623209 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfdc1c-0b1e-4766-b0a1-772e9e3eb2b1 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.623940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcfdc1c-0b1e-4766-b0a1-772e9e3eb2b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:45.625480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcfdc1c-0b1e-4766-b0a1-772e9e3eb2b1 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:45.631657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040476c-fba7-4594-b484-f8fec5e1aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:45.632260 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040476c-fba7-4594-b484-f8fec5e1aa2d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.632535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040476c-fba7-4594-b484-f8fec5e1aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.632804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040476c-fba7-4594-b484-f8fec5e1aa2d None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.633440 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040476c-fba7-4594-b484-f8fec5e1aa2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:45.633745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040476c-fba7-4594-b484-f8fec5e1aa2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:45.635118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040476c-fba7-4594-b484-f8fec5e1aa2d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:45.648569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 173/349] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:45.650529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 177/350] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:45 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:45.651685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcfdc1c-0b1e-4766-b0a1-772e9e3eb2b1 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:45.654588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecde932-4b04-40e2-94cd-b6afe5544922 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.654872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecde932-4b04-40e2-94cd-b6afe5544922 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.655167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecde932-4b04-40e2-94cd-b6afe5544922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.680730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2ecde932-4b04-40e2-94cd-b6afe5544922 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:45.682012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2ecde932-4b04-40e2-94cd-b6afe5544922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:45.776446 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 174/351] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:45.777687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecde932-4b04-40e2-94cd-b6afe5544922 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:45.792024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcfdc1c-0b1e-4766-b0a1-772e9e3eb2b1 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:45.829431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:45.840355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d210e9-f259-4247-b477-7343e7544497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:45.841355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d210e9-f259-4247-b477-7343e7544497 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.841815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d210e9-f259-4247-b477-7343e7544497 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.842231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d210e9-f259-4247-b477-7343e7544497 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.843162 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d210e9-f259-4247-b477-7343e7544497 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:45.845436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d210e9-f259-4247-b477-7343e7544497 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:45.874857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 178/352] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:45 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:45.886777 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:45.933832 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecde932-4b04-40e2-94cd-b6afe5544922 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:45.968856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026093b6-73b5-49f7-9ba2-5e508774e637 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.969204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026093b6-73b5-49f7-9ba2-5e508774e637 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.969431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026093b6-73b5-49f7-9ba2-5e508774e637 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.971752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 179/353] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:45.977768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680a255-dacc-41be-afb6-1d1a63a81ed5 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.978049 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680a255-dacc-41be-afb6-1d1a63a81ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.978372 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680a255-dacc-41be-afb6-1d1a63a81ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.983986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670f32e-37ad-4cd5-a17d-1917c2b4006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:45.984960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670f32e-37ad-4cd5-a17d-1917c2b4006f None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:45.985451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670f32e-37ad-4cd5-a17d-1917c2b4006f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:45.985779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670f32e-37ad-4cd5-a17d-1917c2b4006f None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:45.986446 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670f32e-37ad-4cd5-a17d-1917c2b4006f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:45.987946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670f32e-37ad-4cd5-a17d-1917c2b4006f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:46.000873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a680a255-dacc-41be-afb6-1d1a63a81ed5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:46.003413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a680a255-dacc-41be-afb6-1d1a63a81ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:46.016094 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 175/354] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:45 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:46.095942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 180/355] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:46.110314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670f32e-37ad-4cd5-a17d-1917c2b4006f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:46.230033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670f32e-37ad-4cd5-a17d-1917c2b4006f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:46.248035 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:46.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200bee7-24a4-4c1b-86c7-72d790ffd7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:46.268903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200bee7-24a4-4c1b-86c7-72d790ffd7cd None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:46.269135 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200bee7-24a4-4c1b-86c7-72d790ffd7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:46.269351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200bee7-24a4-4c1b-86c7-72d790ffd7cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:46.281669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200bee7-24a4-4c1b-86c7-72d790ffd7cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:46.283156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200bee7-24a4-4c1b-86c7-72d790ffd7cd None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-5200bee7-24a4-4c1b-86c7-72d790ffd7cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:46.298355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:46.299667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:46.300690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:46.300690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:46.300690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:46.300690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 176/356] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:46.307078 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680a255-dacc-41be-afb6-1d1a63a81ed5 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:46.402508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200bee7-24a4-4c1b-86c7-72d790ffd7cd None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:46.424039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680a255-dacc-41be-afb6-1d1a63a81ed5 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:46.442078 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:46.471495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08714dbb-27be-49f9-a337-094bcbd53418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:46.472449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08714dbb-27be-49f9-a337-094bcbd53418 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:46.473020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08714dbb-27be-49f9-a337-094bcbd53418 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:46.473496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08714dbb-27be-49f9-a337-094bcbd53418 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:46.474334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08714dbb-27be-49f9-a337-094bcbd53418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:46.474826 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08714dbb-27be-49f9-a337-094bcbd53418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:46.476882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08714dbb-27be-49f9-a337-094bcbd53418 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:46.491137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 181/357] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:46.497475 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08714dbb-27be-49f9-a337-094bcbd53418 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:46.570268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200bee7-24a4-4c1b-86c7-72d790ffd7cd None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:46.624370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:46.625330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:46.625850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:46.626328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:46.627284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:46.628924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:46.640825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.sql.core [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 28 18:53:46.640825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 28 18:53:46.640825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [parameters: {'id': '8a7e15c05ec741a3939d1582ee409f3f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 28 18:53:46.640825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81364) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 28 18:53:46.657202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08714dbb-27be-49f9-a337-094bcbd53418 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result.read() Oct 28 18:53:46.671999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return ref.to_dict() Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:53:46.672912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application yield resource Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.session.rollback() Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.force_reraise() Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise self.value Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application session.commit() Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 28 18:53:46.673827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._prepare_impl() Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.session.flush() Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._flush(objects) Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application compat.raise_( Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application flush_context.execute() Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rec.execute(self) Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 28 18:53:46.674755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:53:46.675626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result.read() Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:53:46.676427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [parameters: {'id': '8a7e15c05ec741a3939d1582ee409f3f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:46.677223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 28 18:53:46.678001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:46.685386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 177/358] 10.222.0.58 () {60 vars in 1131 bytes} [Fri Oct 28 18:53:46 2022] POST /identity/v3/roles => generated 144 bytes in 280 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:46.688014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:46.708370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4d9db-6569-4257-a302-91debc6bf97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:46.710029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4d9db-6569-4257-a302-91debc6bf97b None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:46.710306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4d9db-6569-4257-a302-91debc6bf97b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:46.710573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4d9db-6569-4257-a302-91debc6bf97b None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:46.713043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4d9db-6569-4257-a302-91debc6bf97b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:46.715293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4d9db-6569-4257-a302-91debc6bf97b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:46.761411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 182/359] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:46 2022] POST /identity/v3/users => generated 249 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:46.764061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:46.899343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b78fa-3a82-49e6-a205-027dd2de6bd1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:46.941823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa1800-7cba-47ca-8ff3-fd238e895168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:46.942781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa1800-7cba-47ca-8ff3-fd238e895168 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:46.943285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa1800-7cba-47ca-8ff3-fd238e895168 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:46.943750 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa1800-7cba-47ca-8ff3-fd238e895168 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:46.955332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa1800-7cba-47ca-8ff3-fd238e895168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-19aa1800-7cba-47ca-8ff3-fd238e895168 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:46.961300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:46.962202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 18:53:46.963003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:46.969065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 178/360] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:53:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:46.976161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4d9db-6569-4257-a302-91debc6bf97b None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:47.033341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:47.112196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4d9db-6569-4257-a302-91debc6bf97b None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:47.149145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f07a0-707d-41f3-ad7f-d221453c5eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:47.149758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f07a0-707d-41f3-ad7f-d221453c5eda None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:47.150036 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f07a0-707d-41f3-ad7f-d221453c5eda None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:47.150304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f07a0-707d-41f3-ad7f-d221453c5eda None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:47.151050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f07a0-707d-41f3-ad7f-d221453c5eda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:47.151355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f07a0-707d-41f3-ad7f-d221453c5eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:47.152766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f07a0-707d-41f3-ad7f-d221453c5eda None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:47.165388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 183/361] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:48.484992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1e6d1-f75f-4d33-91bd-ca84ba4ecb13 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:48.485550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1e6d1-f75f-4d33-91bd-ca84ba4ecb13 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:48.485550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1e6d1-f75f-4d33-91bd-ca84ba4ecb13 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:48.487177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 179/362] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:48.492564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53e930-2939-4c34-84d4-979f25309e13 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:48.492989 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53e930-2939-4c34-84d4-979f25309e13 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:48.493185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53e930-2939-4c34-84d4-979f25309e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:48.498433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec03c3e-0f4b-4ed7-9d93-ab9c96c833a3 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:48.498718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec03c3e-0f4b-4ed7-9d93-ab9c96c833a3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:48.499017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec03c3e-0f4b-4ed7-9d93-ab9c96c833a3 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:48.500518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 180/363] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:48.516843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad3e5d-d904-4e49-ae01-757297e88e79 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:48.517140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad3e5d-d904-4e49-ae01-757297e88e79 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:48.517399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad3e5d-d904-4e49-ae01-757297e88e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:48.532158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-af53e930-2939-4c34-84d4-979f25309e13 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:48.533482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-af53e930-2939-4c34-84d4-979f25309e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:48.546444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-c5ad3e5d-d904-4e49-ae01-757297e88e79 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:48.547652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c5ad3e5d-d904-4e49-ae01-757297e88e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:48.654347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 181/364] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:48.656806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 184/365] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:48.951643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368f240-571d-4d80-9a4c-f1f21ae69507 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:48.951980 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368f240-571d-4d80-9a4c-f1f21ae69507 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:48.952182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368f240-571d-4d80-9a4c-f1f21ae69507 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:48.953535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 182/366] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:48.958609 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008ce3-dc0a-4f06-b24a-ebd2f89afc67 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:48.958893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008ce3-dc0a-4f06-b24a-ebd2f89afc67 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:48.959198 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008ce3-dc0a-4f06-b24a-ebd2f89afc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:48.967352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368f240-571d-4d80-9a4c-f1f21ae69507 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:48.987346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f3008ce3-dc0a-4f06-b24a-ebd2f89afc67 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:48.988541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f3008ce3-dc0a-4f06-b24a-ebd2f89afc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:49.000445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:49.007299 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:49.068737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 185/367] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:49.069946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008ce3-dc0a-4f06-b24a-ebd2f89afc67 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:49.124349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368f240-571d-4d80-9a4c-f1f21ae69507 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:49.196246 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f455-711c-4279-9a84-4fb6d1e967a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:49.196876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f455-711c-4279-9a84-4fb6d1e967a1 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.197151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f455-711c-4279-9a84-4fb6d1e967a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.197417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f455-711c-4279-9a84-4fb6d1e967a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.221051 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4f455-711c-4279-9a84-4fb6d1e967a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-19d4f455-711c-4279-9a84-4fb6d1e967a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:49.226889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:49.228205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:49.229236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 183/368] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:49.236160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f455-711c-4279-9a84-4fb6d1e967a1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:49.276241 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008ce3-dc0a-4f06-b24a-ebd2f89afc67 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:49.351396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:49.359556 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da5f07-a9c9-4904-b388-876d51b46535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:49.360193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da5f07-a9c9-4904-b388-876d51b46535 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.360470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da5f07-a9c9-4904-b388-876d51b46535 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.360738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da5f07-a9c9-4904-b388-876d51b46535 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.361324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f455-711c-4279-9a84-4fb6d1e967a1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:49.379695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da5f07-a9c9-4904-b388-876d51b46535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-a7da5f07-a9c9-4904-b388-876d51b46535 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:49.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:49.386895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:49.389104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 186/369] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:49.389821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da5f07-a9c9-4904-b388-876d51b46535 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:49.405095 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8af6d8-f038-4ada-a458-bb7edfb48b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:49.406032 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8af6d8-f038-4ada-a458-bb7edfb48b8d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.406486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8af6d8-f038-4ada-a458-bb7edfb48b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.406940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8af6d8-f038-4ada-a458-bb7edfb48b8d None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.407425 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:49.408999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8af6d8-f038-4ada-a458-bb7edfb48b8d None admin] RBAC: Adding query filter params (name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:49.409558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8af6d8-f038-4ada-a458-bb7edfb48b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:49.411236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8af6d8-f038-4ada-a458-bb7edfb48b8d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:49.421707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 184/370] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:53:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:49.422580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8af6d8-f038-4ada-a458-bb7edfb48b8d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:49.511915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da5f07-a9c9-4904-b388-876d51b46535 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:49.551995 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eca1e-5472-46d7-b4ee-d47d7c101aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:49.552657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eca1e-5472-46d7-b4ee-d47d7c101aad None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.553046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eca1e-5472-46d7-b4ee-d47d7c101aad None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.553409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eca1e-5472-46d7-b4ee-d47d7c101aad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.563286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8af6d8-f038-4ada-a458-bb7edfb48b8d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:49.564818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15eca1e-5472-46d7-b4ee-d47d7c101aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-f15eca1e-5472-46d7-b4ee-d47d7c101aad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:49.570028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:49.571124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:49.572084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:49.573719 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 187/371] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:49.574594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eca1e-5472-46d7-b4ee-d47d7c101aad None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:49.610578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe382-1166-4ccb-b219-5882d58165db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:49.611371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe382-1166-4ccb-b219-5882d58165db None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.611752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe382-1166-4ccb-b219-5882d58165db None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.612024 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe382-1166-4ccb-b219-5882d58165db None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.612688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fe382-1166-4ccb-b219-5882d58165db None admin] RBAC: Adding query filter params (name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:49.612988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fe382-1166-4ccb-b219-5882d58165db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:49.614384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fe382-1166-4ccb-b219-5882d58165db None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:49.623510 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 185/372] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:53:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:49.627385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe382-1166-4ccb-b219-5882d58165db None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:49.631113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:49.656439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:49.691435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eca1e-5472-46d7-b4ee-d47d7c101aad None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:49.726254 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dad96c-40aa-4609-a7e1-1e5ec6aa17d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:49.727170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dad96c-40aa-4609-a7e1-1e5ec6aa17d8 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.727539 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dad96c-40aa-4609-a7e1-1e5ec6aa17d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.727901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dad96c-40aa-4609-a7e1-1e5ec6aa17d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.732855 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe382-1166-4ccb-b219-5882d58165db None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:49.739728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dad96c-40aa-4609-a7e1-1e5ec6aa17d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:49.741411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dad96c-40aa-4609-a7e1-1e5ec6aa17d8 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-e0dad96c-40aa-4609-a7e1-1e5ec6aa17d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:49.758501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:49.759576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:49.760546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:49.760546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:49.760546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:49.766362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 188/373] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:49.766362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3265500-d079-4f75-8f22-d778db914167 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.766362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3265500-d079-4f75-8f22-d778db914167 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.766362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3265500-d079-4f75-8f22-d778db914167 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.766362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 189/374] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:49.766362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3265500-d079-4f75-8f22-d778db914167 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:49.777567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842dd48-3ee3-48a7-a2d4-014d6dec1e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:49.778331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842dd48-3ee3-48a7-a2d4-014d6dec1e64 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.778709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842dd48-3ee3-48a7-a2d4-014d6dec1e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.779110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842dd48-3ee3-48a7-a2d4-014d6dec1e64 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.779871 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842dd48-3ee3-48a7-a2d4-014d6dec1e64 None admin] RBAC: Adding query filter params (name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:49.780307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842dd48-3ee3-48a7-a2d4-014d6dec1e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:49.781704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842dd48-3ee3-48a7-a2d4-014d6dec1e64 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:49.789912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 186/375] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:53:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:49.795436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842dd48-3ee3-48a7-a2d4-014d6dec1e64 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:49.863099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:49.874639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3265500-d079-4f75-8f22-d778db914167 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:49.904606 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:49.907498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842dd48-3ee3-48a7-a2d4-014d6dec1e64 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:49.916727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8256a-abbd-4dbf-82da-1338a090ada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:49.917368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8256a-abbd-4dbf-82da-1338a090ada7 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.917645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8256a-abbd-4dbf-82da-1338a090ada7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.917916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8256a-abbd-4dbf-82da-1338a090ada7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.929911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8256a-abbd-4dbf-82da-1338a090ada7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:49.931390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8256a-abbd-4dbf-82da-1338a090ada7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:49.946406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27433746-13a1-4ac2-8315-a20f7e546506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:49.947107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27433746-13a1-4ac2-8315-a20f7e546506 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-f1f8256a-abbd-4dbf-82da-1338a090ada7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:49.947384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:49.948437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:49.949568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:49.949568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:49.949568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:49.949763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27433746-13a1-4ac2-8315-a20f7e546506 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.949763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27433746-13a1-4ac2-8315-a20f7e546506 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.949763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27433746-13a1-4ac2-8315-a20f7e546506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:49.950203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27433746-13a1-4ac2-8315-a20f7e546506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:49.951591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27433746-13a1-4ac2-8315-a20f7e546506 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:49.953214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 190/376] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:49.954441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53f783-4e84-4bcd-98e1-7200b4949e1d None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:49.954729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53f783-4e84-4bcd-98e1-7200b4949e1d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:49.955018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53f783-4e84-4bcd-98e1-7200b4949e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:49.968970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 187/377] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:49.970095 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27433746-13a1-4ac2-8315-a20f7e546506 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:49.972733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-4c53f783-4e84-4bcd-98e1-7200b4949e1d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:49.973910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4c53f783-4e84-4bcd-98e1-7200b4949e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:50.042227 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 191/378] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:50.043066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53f783-4e84-4bcd-98e1-7200b4949e1d None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.075442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27433746-13a1-4ac2-8315-a20f7e546506 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.101996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:50.113555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2681e-d4a9-421a-bf4a-b461256a5020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.114226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2681e-d4a9-421a-bf4a-b461256a5020 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.114577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2681e-d4a9-421a-bf4a-b461256a5020 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.114918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2681e-d4a9-421a-bf4a-b461256a5020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.126074 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2681e-d4a9-421a-bf4a-b461256a5020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.127657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2681e-d4a9-421a-bf4a-b461256a5020 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-23d2681e-d4a9-421a-bf4a-b461256a5020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:50.142249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:50.143142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:50.149070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:50.149070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:50.149070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:50.149070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 188/379] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:50.149290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53f783-4e84-4bcd-98e1-7200b4949e1d None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.149364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2681e-d4a9-421a-bf4a-b461256a5020 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.152197 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:50.184075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd1712-4aa2-4872-9d09-e380073a8400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.184709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd1712-4aa2-4872-9d09-e380073a8400 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.184984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd1712-4aa2-4872-9d09-e380073a8400 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.185256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd1712-4aa2-4872-9d09-e380073a8400 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.185894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd1712-4aa2-4872-9d09-e380073a8400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:50.186195 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd1712-4aa2-4872-9d09-e380073a8400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.187588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd1712-4aa2-4872-9d09-e380073a8400 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:50.199357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 192/380] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:50.204766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc8b1-8d20-438a-85c4-f7acdc852ded None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.205049 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc8b1-8d20-438a-85c4-f7acdc852ded None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.205321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc8b1-8d20-438a-85c4-f7acdc852ded None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.206563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 193/381] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:50.210782 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc8b1-8d20-438a-85c4-f7acdc852ded None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.257812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2681e-d4a9-421a-bf4a-b461256a5020 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.293935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f626682-c62a-415d-926e-88dd92b75a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.295110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f626682-c62a-415d-926e-88dd92b75a14 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.295449 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f626682-c62a-415d-926e-88dd92b75a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.295728 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f626682-c62a-415d-926e-88dd92b75a14 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.303385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f626682-c62a-415d-926e-88dd92b75a14 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.304870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f626682-c62a-415d-926e-88dd92b75a14 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-2f626682-c62a-415d-926e-88dd92b75a14 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:53:50.317590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:50.318656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:53:50.319781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:50.320440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc8b1-8d20-438a-85c4-f7acdc852ded None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.323138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 189/382] 10.222.0.58 () {56 vars in 1093 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:50.324054 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f626682-c62a-415d-926e-88dd92b75a14 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.353544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b3d4-56f0-4a64-8941-fecd48d54f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.354143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b3d4-56f0-4a64-8941-fecd48d54f88 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.354417 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b3d4-56f0-4a64-8941-fecd48d54f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.354685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b3d4-56f0-4a64-8941-fecd48d54f88 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.355340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03b3d4-56f0-4a64-8941-fecd48d54f88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:50.355640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03b3d4-56f0-4a64-8941-fecd48d54f88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.357564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03b3d4-56f0-4a64-8941-fecd48d54f88 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:50.364498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:50.368598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 194/383] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:50.369984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4432e-a9d1-4dee-bdc9-8c257a229859 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.370269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4432e-a9d1-4dee-bdc9-8c257a229859 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.370540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4432e-a9d1-4dee-bdc9-8c257a229859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.388678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e6d4432e-a9d1-4dee-bdc9-8c257a229859 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:50.389850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e6d4432e-a9d1-4dee-bdc9-8c257a229859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:50.410055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:50.427582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f626682-c62a-415d-926e-88dd92b75a14 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.455196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 195/384] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:50.455918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4432e-a9d1-4dee-bdc9-8c257a229859 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.474640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92646a4f-adff-4044-825f-65028a701a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.475297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92646a4f-adff-4044-825f-65028a701a1d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.475574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92646a4f-adff-4044-825f-65028a701a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.475842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92646a4f-adff-4044-825f-65028a701a1d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.484530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92646a4f-adff-4044-825f-65028a701a1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.486072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92646a4f-adff-4044-825f-65028a701a1d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-92646a4f-adff-4044-825f-65028a701a1d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:53:50.497295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:50.498358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 28 18:53:50.499466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:50.500368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 190/385] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:50.500368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92646a4f-adff-4044-825f-65028a701a1d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.567976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4432e-a9d1-4dee-bdc9-8c257a229859 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.602881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:50.603498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad231831-2c56-4283-83b6-27b644582b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.604116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad231831-2c56-4283-83b6-27b644582b7e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.604390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad231831-2c56-4283-83b6-27b644582b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.604665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad231831-2c56-4283-83b6-27b644582b7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.607529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92646a4f-adff-4044-825f-65028a701a1d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.616391 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad231831-2c56-4283-83b6-27b644582b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-ad231831-2c56-4283-83b6-27b644582b7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:50.621558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:50.622542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 196/386] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:50.627851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad231831-2c56-4283-83b6-27b644582b7e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.644125 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:50.649627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3c334-f0f0-4fa6-ad27-233364432e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.650233 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3c334-f0f0-4fa6-ad27-233364432e93 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.650511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3c334-f0f0-4fa6-ad27-233364432e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.650778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3c334-f0f0-4fa6-ad27-233364432e93 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.651528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3c334-f0f0-4fa6-ad27-233364432e93 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:50.651834 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3c334-f0f0-4fa6-ad27-233364432e93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.653821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3c334-f0f0-4fa6-ad27-233364432e93 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:50.689196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 191/387] 10.222.0.58 () {56 vars in 1133 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:50.690010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3c334-f0f0-4fa6-ad27-233364432e93 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.738585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad231831-2c56-4283-83b6-27b644582b7e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.773068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc145ec-70f0-499c-9b2e-ba5ca273641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.773687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc145ec-70f0-499c-9b2e-ba5ca273641f None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.773965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc145ec-70f0-499c-9b2e-ba5ca273641f None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.774233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc145ec-70f0-499c-9b2e-ba5ca273641f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.781146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc145ec-70f0-499c-9b2e-ba5ca273641f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.782578 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc145ec-70f0-499c-9b2e-ba5ca273641f None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:50.790718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3c334-f0f0-4fa6-ad27-233364432e93 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-8bc145ec-70f0-499c-9b2e-ba5ca273641f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:53:50.795413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:50.796554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 197/388] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:50.797707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc145ec-70f0-499c-9b2e-ba5ca273641f None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.826882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707ca9b-c327-4a8c-8edf-a1602d7cfa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.827513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707ca9b-c327-4a8c-8edf-a1602d7cfa6a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.827785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707ca9b-c327-4a8c-8edf-a1602d7cfa6a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.828050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707ca9b-c327-4a8c-8edf-a1602d7cfa6a None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.828764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707ca9b-c327-4a8c-8edf-a1602d7cfa6a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:50.829061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707ca9b-c327-4a8c-8edf-a1602d7cfa6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.830485 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707ca9b-c327-4a8c-8edf-a1602d7cfa6a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:50.847213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 192/389] 10.222.0.58 () {56 vars in 1137 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:50.848114 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707ca9b-c327-4a8c-8edf-a1602d7cfa6a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.865770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:50.876002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:50.899739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc145ec-70f0-499c-9b2e-ba5ca273641f None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.931664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8a10-7d11-4b7d-9f9e-19108c698ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.932259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8a10-7d11-4b7d-9f9e-19108c698ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.932531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8a10-7d11-4b7d-9f9e-19108c698ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.932797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8a10-7d11-4b7d-9f9e-19108c698ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.933455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8a10-7d11-4b7d-9f9e-19108c698ba2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:50.933753 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8a10-7d11-4b7d-9f9e-19108c698ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.935700 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8a10-7d11-4b7d-9f9e-19108c698ba2 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:50.943306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 198/390] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:50.947077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8a10-7d11-4b7d-9f9e-19108c698ba2 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:50.949280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707ca9b-c327-4a8c-8edf-a1602d7cfa6a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:50.981529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1733724-9ea1-4d5f-a0fb-b24c19eb4025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:50.982142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1733724-9ea1-4d5f-a0fb-b24c19eb4025 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:50.982418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1733724-9ea1-4d5f-a0fb-b24c19eb4025 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:50.982683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1733724-9ea1-4d5f-a0fb-b24c19eb4025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:50.992526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1733724-9ea1-4d5f-a0fb-b24c19eb4025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:50.993964 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1733724-9ea1-4d5f-a0fb-b24c19eb4025 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-c1733724-9ea1-4d5f-a0fb-b24c19eb4025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:51.009537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:51.010581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:51.011453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:51.011453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.011453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:51.011453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 193/391] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:51.011692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1733724-9ea1-4d5f-a0fb-b24c19eb4025 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.047242 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8a10-7d11-4b7d-9f9e-19108c698ba2 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.073867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:51.079839 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79767dc-b39b-48bf-91a8-c2bde0c1e78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.080464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79767dc-b39b-48bf-91a8-c2bde0c1e78a None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.080738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79767dc-b39b-48bf-91a8-c2bde0c1e78a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.081006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79767dc-b39b-48bf-91a8-c2bde0c1e78a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.088462 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:51.092296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79767dc-b39b-48bf-91a8-c2bde0c1e78a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.093941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79767dc-b39b-48bf-91a8-c2bde0c1e78a None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-e79767dc-b39b-48bf-91a8-c2bde0c1e78a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:51.108185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:51.109059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:51.109897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:51.109897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.109897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:51.110056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1733724-9ea1-4d5f-a0fb-b24c19eb4025 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.110126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 199/392] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:51.110126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79767dc-b39b-48bf-91a8-c2bde0c1e78a None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.143889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dba74b-6310-46c5-bc27-16a16fbc0a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.144416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba74b-6310-46c5-bc27-16a16fbc0a4a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.144690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba74b-6310-46c5-bc27-16a16fbc0a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.144950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba74b-6310-46c5-bc27-16a16fbc0a4a None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.145665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dba74b-6310-46c5-bc27-16a16fbc0a4a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:51.145956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dba74b-6310-46c5-bc27-16a16fbc0a4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.147399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dba74b-6310-46c5-bc27-16a16fbc0a4a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.163877 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 194/393] 10.222.0.58 () {56 vars in 1137 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:51.164650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dba74b-6310-46c5-bc27-16a16fbc0a4a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.212738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79767dc-b39b-48bf-91a8-c2bde0c1e78a None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.245714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74306ba7-cb5b-4487-a696-7d2d3831ec21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.246320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74306ba7-cb5b-4487-a696-7d2d3831ec21 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.246593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74306ba7-cb5b-4487-a696-7d2d3831ec21 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.246861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74306ba7-cb5b-4487-a696-7d2d3831ec21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.257565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74306ba7-cb5b-4487-a696-7d2d3831ec21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.258990 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74306ba7-cb5b-4487-a696-7d2d3831ec21 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.261542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dba74b-6310-46c5-bc27-16a16fbc0a4a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-74306ba7-cb5b-4487-a696-7d2d3831ec21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:51.273442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:51.274488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:51.280107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:51.280107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.280107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:51.280107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 200/394] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:51.280107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74306ba7-cb5b-4487-a696-7d2d3831ec21 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.284843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:51.296700 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06718df7-b8f3-4db8-bf9f-fe7ee5f647c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.297320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06718df7-b8f3-4db8-bf9f-fe7ee5f647c1 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.297592 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06718df7-b8f3-4db8-bf9f-fe7ee5f647c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.297863 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06718df7-b8f3-4db8-bf9f-fe7ee5f647c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.307632 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06718df7-b8f3-4db8-bf9f-fe7ee5f647c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.309191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06718df7-b8f3-4db8-bf9f-fe7ee5f647c1 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.320432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-06718df7-b8f3-4db8-bf9f-fe7ee5f647c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:51.324821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:51.325866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:51.331545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:51.331545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.331545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:51.331545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 195/395] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:51.331545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06718df7-b8f3-4db8-bf9f-fe7ee5f647c1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.378482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74306ba7-cb5b-4487-a696-7d2d3831ec21 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.416292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa7e39-6632-4266-a189-4b873a46759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.417115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa7e39-6632-4266-a189-4b873a46759b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.417602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa7e39-6632-4266-a189-4b873a46759b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.418029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa7e39-6632-4266-a189-4b873a46759b None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.419807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa7e39-6632-4266-a189-4b873a46759b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:51.420286 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa7e39-6632-4266-a189-4b873a46759b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.422081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa7e39-6632-4266-a189-4b873a46759b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.426382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06718df7-b8f3-4db8-bf9f-fe7ee5f647c1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.436436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 201/396] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:51.437638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa7e39-6632-4266-a189-4b873a46759b None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.458934 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720aec1-a51d-434b-b312-65cdfe6e32c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.459565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720aec1-a51d-434b-b312-65cdfe6e32c5 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.459844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720aec1-a51d-434b-b312-65cdfe6e32c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.460116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720aec1-a51d-434b-b312-65cdfe6e32c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.460758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720aec1-a51d-434b-b312-65cdfe6e32c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:51.461061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720aec1-a51d-434b-b312-65cdfe6e32c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.462434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720aec1-a51d-434b-b312-65cdfe6e32c5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.474128 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 196/397] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:51.475084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720aec1-a51d-434b-b312-65cdfe6e32c5 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.509735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:51.546080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa7e39-6632-4266-a189-4b873a46759b None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.547709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:51.572027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720aec1-a51d-434b-b312-65cdfe6e32c5 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.580120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b5cf3-80be-4844-9ffa-fc107035cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.580732 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b5cf3-80be-4844-9ffa-fc107035cdc9 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.581014 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b5cf3-80be-4844-9ffa-fc107035cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.581284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b5cf3-80be-4844-9ffa-fc107035cdc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.591441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b5cf3-80be-4844-9ffa-fc107035cdc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.593177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b5cf3-80be-4844-9ffa-fc107035cdc9 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-974b5cf3-80be-4844-9ffa-fc107035cdc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:51.608647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:51.609641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:51.610752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:51.610752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:51.610752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:51.611711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44713dbb-e4db-4771-99ea-3c1455223814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.612326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44713dbb-e4db-4771-99ea-3c1455223814 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.612604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44713dbb-e4db-4771-99ea-3c1455223814 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.612876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44713dbb-e4db-4771-99ea-3c1455223814 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.613505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44713dbb-e4db-4771-99ea-3c1455223814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:51.613803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44713dbb-e4db-4771-99ea-3c1455223814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.614618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 202/398] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:51.615357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44713dbb-e4db-4771-99ea-3c1455223814 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.615457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b5cf3-80be-4844-9ffa-fc107035cdc9 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.626827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 197/399] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:51.627717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44713dbb-e4db-4771-99ea-3c1455223814 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.718268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b5cf3-80be-4844-9ffa-fc107035cdc9 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.720969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:51.733507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44713dbb-e4db-4771-99ea-3c1455223814 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.753110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54656d7-fadc-493b-8bf1-7acf7084f775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.753706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54656d7-fadc-493b-8bf1-7acf7084f775 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.753983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54656d7-fadc-493b-8bf1-7acf7084f775 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.754249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54656d7-fadc-493b-8bf1-7acf7084f775 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.754880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54656d7-fadc-493b-8bf1-7acf7084f775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:51.755206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54656d7-fadc-493b-8bf1-7acf7084f775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.756599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54656d7-fadc-493b-8bf1-7acf7084f775 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.767505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8ac4e-c3f2-4965-bce4-a068e84a3c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.768021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 203/400] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:51.768137 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8ac4e-c3f2-4965-bce4-a068e84a3c8b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.768662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8ac4e-c3f2-4965-bce4-a068e84a3c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.768931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8ac4e-c3f2-4965-bce4-a068e84a3c8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.769136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54656d7-fadc-493b-8bf1-7acf7084f775 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.775563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8ac4e-c3f2-4965-bce4-a068e84a3c8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.777010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8ac4e-c3f2-4965-bce4-a068e84a3c8b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.785612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-2cd8ac4e-c3f2-4965-bce4-a068e84a3c8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:53:51.789332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:51.790322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 198/401] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:51.791321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8ac4e-c3f2-4965-bce4-a068e84a3c8b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.873415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54656d7-fadc-493b-8bf1-7acf7084f775 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.889573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8ac4e-c3f2-4965-bce4-a068e84a3c8b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:51.907616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032ec55-56b7-4161-a6f9-83ec596390dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.908216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032ec55-56b7-4161-a6f9-83ec596390dd None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.908496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032ec55-56b7-4161-a6f9-83ec596390dd None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.908770 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032ec55-56b7-4161-a6f9-83ec596390dd None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.909406 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032ec55-56b7-4161-a6f9-83ec596390dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.910829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032ec55-56b7-4161-a6f9-83ec596390dd None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.922508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418cf79-8178-4588-8133-71894b9ca8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:51.923144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418cf79-8178-4588-8133-71894b9ca8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:51.923474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418cf79-8178-4588-8133-71894b9ca8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:51.923788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418cf79-8178-4588-8133-71894b9ca8c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:51.924447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418cf79-8178-4588-8133-71894b9ca8c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:51.924782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418cf79-8178-4588-8133-71894b9ca8c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:51.926142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418cf79-8178-4588-8133-71894b9ca8c4 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:51.931619 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 204/402] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:53:51 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:53:51.932422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032ec55-56b7-4161-a6f9-83ec596390dd None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.942017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 199/403] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:51.944096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418cf79-8178-4588-8133-71894b9ca8c4 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:51.967912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:52.017478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:52.053668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032ec55-56b7-4161-a6f9-83ec596390dd None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.061191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418cf79-8178-4588-8133-71894b9ca8c4 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.094070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8216ee8-e7e5-46bc-98bf-eff5a36ab3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.095055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8216ee8-e7e5-46bc-98bf-eff5a36ab3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.095294 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8216ee8-e7e5-46bc-98bf-eff5a36ab3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.095567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8216ee8-e7e5-46bc-98bf-eff5a36ab3d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.098281 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e6ea3-3cad-40cd-82f1-591ca3d47683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.098915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e6ea3-3cad-40cd-82f1-591ca3d47683 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.099213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e6ea3-3cad-40cd-82f1-591ca3d47683 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.099483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e6ea3-3cad-40cd-82f1-591ca3d47683 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.103354 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8216ee8-e7e5-46bc-98bf-eff5a36ab3d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.104815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8216ee8-e7e5-46bc-98bf-eff5a36ab3d6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.107509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e6ea3-3cad-40cd-82f1-591ca3d47683 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.108935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e6ea3-3cad-40cd-82f1-591ca3d47683 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-e8216ee8-e7e5-46bc-98bf-eff5a36ab3d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:53:52.118340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:52.119211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:52.120114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:53:52.120114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:53:52.120114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:53:52.120114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:53:52.120114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:53:52.120114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:53:52.120114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:52.120114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:52.120926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 205/404] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-9a5e6ea3-3cad-40cd-82f1-591ca3d47683 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:53:52.121028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:52.121835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 18:53:52.122753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:52.123924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8216ee8-e7e5-46bc-98bf-eff5a36ab3d6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:52.124088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 200/405] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:52.124088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e6ea3-3cad-40cd-82f1-591ca3d47683 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:52.204202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:52.228476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e6ea3-3cad-40cd-82f1-591ca3d47683 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.237316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8216ee8-e7e5-46bc-98bf-eff5a36ab3d6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.265516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533a01c-3420-43aa-ac26-812dd6caa9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.266136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533a01c-3420-43aa-ac26-812dd6caa9de None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.266413 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533a01c-3420-43aa-ac26-812dd6caa9de None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.266679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533a01c-3420-43aa-ac26-812dd6caa9de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.267450 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:52.273792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533a01c-3420-43aa-ac26-812dd6caa9de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.274282 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965853a4-4e9d-4fc7-94a8-9953f553d406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.274892 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965853a4-4e9d-4fc7-94a8-9953f553d406 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.275201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965853a4-4e9d-4fc7-94a8-9953f553d406 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.275295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533a01c-3420-43aa-ac26-812dd6caa9de None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.275476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965853a4-4e9d-4fc7-94a8-9953f553d406 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.276174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965853a4-4e9d-4fc7-94a8-9953f553d406 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:52.276481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965853a4-4e9d-4fc7-94a8-9953f553d406 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.278457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965853a4-4e9d-4fc7-94a8-9953f553d406 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-3533a01c-3420-43aa-ac26-812dd6caa9de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:53:52.286535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:52.287364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 201/406] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:52.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533a01c-3420-43aa-ac26-812dd6caa9de None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:52.296968 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 206/407] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:52.297804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965853a4-4e9d-4fc7-94a8-9953f553d406 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:52.398283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533a01c-3420-43aa-ac26-812dd6caa9de None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.406011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965853a4-4e9d-4fc7-94a8-9953f553d406 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.435827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd86f3b-4cd7-4d9a-ace1-faa2589a67b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.436426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd86f3b-4cd7-4d9a-ace1-faa2589a67b1 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.436695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd86f3b-4cd7-4d9a-ace1-faa2589a67b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.436963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd86f3b-4cd7-4d9a-ace1-faa2589a67b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.437644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd86f3b-4cd7-4d9a-ace1-faa2589a67b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:52.437946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd86f3b-4cd7-4d9a-ace1-faa2589a67b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.439412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd86f3b-4cd7-4d9a-ace1-faa2589a67b1 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.443225 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:52.444130 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6416bf-da44-4d87-b1a5-fd71b7b59d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.445048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6416bf-da44-4d87-b1a5-fd71b7b59d80 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.445540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6416bf-da44-4d87-b1a5-fd71b7b59d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.446023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6416bf-da44-4d87-b1a5-fd71b7b59d80 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.447128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6416bf-da44-4d87-b1a5-fd71b7b59d80 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:52.447629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6416bf-da44-4d87-b1a5-fd71b7b59d80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.449229 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6416bf-da44-4d87-b1a5-fd71b7b59d80 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.458706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 202/408] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:52.459558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd86f3b-4cd7-4d9a-ace1-faa2589a67b1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:52.489900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 207/409] 10.222.0.58 () {56 vars in 1139 bytes} [Fri Oct 28 18:53:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:52.491030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6416bf-da44-4d87-b1a5-fd71b7b59d80 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:52.548593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:52.563519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd86f3b-4cd7-4d9a-ace1-faa2589a67b1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.596915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325f899-4092-4773-8acb-75a54db0aa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.597512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325f899-4092-4773-8acb-75a54db0aa4b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.597796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325f899-4092-4773-8acb-75a54db0aa4b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.598045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325f899-4092-4773-8acb-75a54db0aa4b None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.598724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325f899-4092-4773-8acb-75a54db0aa4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:52.599146 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325f899-4092-4773-8acb-75a54db0aa4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.600464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325f899-4092-4773-8acb-75a54db0aa4b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.610172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6416bf-da44-4d87-b1a5-fd71b7b59d80 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.613077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 203/410] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:52.613567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325f899-4092-4773-8acb-75a54db0aa4b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:52.651261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8529b-d78c-46fc-bfa9-cb461814ccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.651921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8529b-d78c-46fc-bfa9-cb461814ccba None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.652201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8529b-d78c-46fc-bfa9-cb461814ccba None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.652455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8529b-d78c-46fc-bfa9-cb461814ccba None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a758611cf8d94e8ca8aad6a75bef5e34/roles/01b71dd1f22e41269c3caa83dea57b2e` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.668331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8529b-d78c-46fc-bfa9-cb461814ccba None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=a758611cf8d94e8ca8aad6a75bef5e34, role_id=01b71dd1f22e41269c3caa83dea57b2e)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.670320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8529b-d78c-46fc-bfa9-cb461814ccba None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.671786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:52.691090 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 208/411] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:52 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a758611cf8d94e8ca8aad6a75bef5e34/roles/01b71dd1f22e41269c3caa83dea57b2e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:52.692184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8529b-d78c-46fc-bfa9-cb461814ccba None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:52.727992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325f899-4092-4773-8acb-75a54db0aa4b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.783981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b892dbf-2a2c-424c-975f-9dcdff9fcde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.784930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b892dbf-2a2c-424c-975f-9dcdff9fcde5 None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.785254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b892dbf-2a2c-424c-975f-9dcdff9fcde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.785532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b892dbf-2a2c-424c-975f-9dcdff9fcde5 None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/6e8ec16bd9254b23bfd2b60551af9cad/roles/01b71dd1f22e41269c3caa83dea57b2e` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.808930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b892dbf-2a2c-424c-975f-9dcdff9fcde5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=6e8ec16bd9254b23bfd2b60551af9cad, role_id=01b71dd1f22e41269c3caa83dea57b2e)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.811518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b892dbf-2a2c-424c-975f-9dcdff9fcde5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.827526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:52.833373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 204/412] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:52 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/6e8ec16bd9254b23bfd2b60551af9cad/roles/01b71dd1f22e41269c3caa83dea57b2e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:52.836200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8529b-d78c-46fc-bfa9-cb461814ccba None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:52.837777 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b892dbf-2a2c-424c-975f-9dcdff9fcde5 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:52.900697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1005af-3918-4470-b175-655311103c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:52.901417 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1005af-3918-4470-b175-655311103c71 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:52.901695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1005af-3918-4470-b175-655311103c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:52.901970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1005af-3918-4470-b175-655311103c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:52.916847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1005af-3918-4470-b175-655311103c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:52.918478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1005af-3918-4470-b175-655311103c71 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-3a1005af-3918-4470-b175-655311103c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:52.937346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:52.938310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:52.939389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:52.939389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:52.939389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:52.939389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 209/413] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:52.944336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1005af-3918-4470-b175-655311103c71 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:53.003233 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b892dbf-2a2c-424c-975f-9dcdff9fcde5 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:53.012407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:53.071867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2a250-9460-46c3-84c7-150158577ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:53.072452 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2a250-9460-46c3-84c7-150158577ebc None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:53.072728 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2a250-9460-46c3-84c7-150158577ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:53.072998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2a250-9460-46c3-84c7-150158577ebc None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/302ded4fdd72405c8ce8a364f7d2400c/roles/01b71dd1f22e41269c3caa83dea57b2e` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:53.073416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1005af-3918-4470-b175-655311103c71 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:53.092135 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2a250-9460-46c3-84c7-150158577ebc None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=302ded4fdd72405c8ce8a364f7d2400c, role_id=01b71dd1f22e41269c3caa83dea57b2e)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:53.094182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2a250-9460-46c3-84c7-150158577ebc None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:53.116078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 205/414] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:52 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/302ded4fdd72405c8ce8a364f7d2400c/roles/01b71dd1f22e41269c3caa83dea57b2e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:53.121326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd672164-c37f-487a-a3c2-cb0c30531d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:53.122087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd672164-c37f-487a-a3c2-cb0c30531d42 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:53.122367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd672164-c37f-487a-a3c2-cb0c30531d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:53.122677 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd672164-c37f-487a-a3c2-cb0c30531d42 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:53.123399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd672164-c37f-487a-a3c2-cb0c30531d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:53.123702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd672164-c37f-487a-a3c2-cb0c30531d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:53.125203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd672164-c37f-487a-a3c2-cb0c30531d42 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:53.140168 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 210/415] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:53.145748 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2a250-9460-46c3-84c7-150158577ebc None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:53.278377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2a250-9460-46c3-84c7-150158577ebc None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:53.332707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d6aed-7b91-4a42-b01e-1d39b50c07ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:53.333705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d6aed-7b91-4a42-b01e-1d39b50c07ef None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:53.334198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d6aed-7b91-4a42-b01e-1d39b50c07ef None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:53.334662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d6aed-7b91-4a42-b01e-1d39b50c07ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:53.348406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d6aed-7b91-4a42-b01e-1d39b50c07ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:53.350259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d6aed-7b91-4a42-b01e-1d39b50c07ef None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-642d6aed-7b91-4a42-b01e-1d39b50c07ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:53:53.366762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:53.367682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:53.368531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:53:53.368531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:53:53.368531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:53:53.368531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:53:53.368531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:53:53.368531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:53:53.368531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:53.368531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:53.371081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 206/416] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:53.379033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd672164-c37f-487a-a3c2-cb0c30531d42 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:53.397254 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:53.503382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd672164-c37f-487a-a3c2-cb0c30531d42 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:53.542948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73a4f6-1ebc-41c8-8132-45928da96b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:53.543585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73a4f6-1ebc-41c8-8132-45928da96b1b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:53.544399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73a4f6-1ebc-41c8-8132-45928da96b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:53.544728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73a4f6-1ebc-41c8-8132-45928da96b1b None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:53.545420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73a4f6-1ebc-41c8-8132-45928da96b1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:53.545724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73a4f6-1ebc-41c8-8132-45928da96b1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:53.548684 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73a4f6-1ebc-41c8-8132-45928da96b1b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:53.561921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 211/417] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:53:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:53.567347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d6aed-7b91-4a42-b01e-1d39b50c07ef None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:53.595029 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:53.701184 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d6aed-7b91-4a42-b01e-1d39b50c07ef None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:53.741212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9a52-1351-4598-aac3-f700a53dd52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:53.741879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9a52-1351-4598-aac3-f700a53dd52b None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:53.742161 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9a52-1351-4598-aac3-f700a53dd52b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:53.742431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9a52-1351-4598-aac3-f700a53dd52b None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/9cffb662f4104e349c81c578bf66ab7b/roles/01b71dd1f22e41269c3caa83dea57b2e` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:53.763003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9a52-1351-4598-aac3-f700a53dd52b None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=9cffb662f4104e349c81c578bf66ab7b, role_id=01b71dd1f22e41269c3caa83dea57b2e)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:53.764460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9a52-1351-4598-aac3-f700a53dd52b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:53.793031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 207/418] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:53 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/9cffb662f4104e349c81c578bf66ab7b/roles/01b71dd1f22e41269c3caa83dea57b2e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:54.332075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f6fcd-88a1-441d-a682-0d59e2b8583c None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:54.332592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f6fcd-88a1-441d-a682-0d59e2b8583c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:54.332592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f6fcd-88a1-441d-a682-0d59e2b8583c None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:54.334265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 212/419] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:54.339577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a59921-ff9e-4699-ba36-5f0acde9ab99 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:54.339867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a59921-ff9e-4699-ba36-5f0acde9ab99 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:54.340138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a59921-ff9e-4699-ba36-5f0acde9ab99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:54.359360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-41a59921-ff9e-4699-ba36-5f0acde9ab99 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:54.360557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-41a59921-ff9e-4699-ba36-5f0acde9ab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:54.427305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:54.437245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 208/420] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:54.737459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f6fcd-88a1-441d-a682-0d59e2b8583c None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:54.811660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:54.844855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f6fcd-88a1-441d-a682-0d59e2b8583c None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:54.879757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f865a1-b273-49ac-8219-87cac255ee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:54.880384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f865a1-b273-49ac-8219-87cac255ee30 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:54.880660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f865a1-b273-49ac-8219-87cac255ee30 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:54.880926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f865a1-b273-49ac-8219-87cac255ee30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:54.891437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f865a1-b273-49ac-8219-87cac255ee30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-01f865a1-b273-49ac-8219-87cac255ee30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:54.896901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:54.897777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:54.899525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 213/421] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:54.904214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a59921-ff9e-4699-ba36-5f0acde9ab99 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:55.067928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a59921-ff9e-4699-ba36-5f0acde9ab99 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:55.120985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94890ae-5903-4a5f-8097-f9bb47b8ced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:55.122515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94890ae-5903-4a5f-8097-f9bb47b8ced7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:55.123010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94890ae-5903-4a5f-8097-f9bb47b8ced7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:55.123495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94890ae-5903-4a5f-8097-f9bb47b8ced7 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:55.124353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94890ae-5903-4a5f-8097-f9bb47b8ced7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:55.124841 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94890ae-5903-4a5f-8097-f9bb47b8ced7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:55.126385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94890ae-5903-4a5f-8097-f9bb47b8ced7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:55.138026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 209/422] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:53:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:55.143824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f865a1-b273-49ac-8219-87cac255ee30 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:55.261424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f865a1-b273-49ac-8219-87cac255ee30 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:55.292208 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:55.300628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9a5d0-fd0b-4fab-93f9-e1c35c6f66da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:55.301260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9a5d0-fd0b-4fab-93f9-e1c35c6f66da None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:55.301536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9a5d0-fd0b-4fab-93f9-e1c35c6f66da None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:55.301806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9a5d0-fd0b-4fab-93f9-e1c35c6f66da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:55.314353 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9a5d0-fd0b-4fab-93f9-e1c35c6f66da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:55.315881 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9a5d0-fd0b-4fab-93f9-e1c35c6f66da None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-f6d9a5d0-fd0b-4fab-93f9-e1c35c6f66da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:55.332111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:55.333451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:55.334695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:55.334695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:55.334695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:55.334695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 214/423] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:55.339130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94890ae-5903-4a5f-8097-f9bb47b8ced7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:55.362100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1abd7-5bea-4979-9b95-e096c56036ed None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:55.362379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1abd7-5bea-4979-9b95-e096c56036ed None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:55.363193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1abd7-5bea-4979-9b95-e096c56036ed None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:55.365086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 215/424] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:55.369903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff4f5-6139-4073-b1af-74b93921d0ef None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:55.370438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff4f5-6139-4073-b1af-74b93921d0ef None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:55.370676 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff4f5-6139-4073-b1af-74b93921d0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:55.392209 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-404ff4f5-6139-4073-b1af-74b93921d0ef None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:55.394517 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-404ff4f5-6139-4073-b1af-74b93921d0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:55.471752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:55.519961 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 216/425] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:55.531625 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94890ae-5903-4a5f-8097-f9bb47b8ced7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:55.571870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f90e5-75b6-4889-b453-02fbb1534d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:55.572768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f90e5-75b6-4889-b453-02fbb1534d8c None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:55.573192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f90e5-75b6-4889-b453-02fbb1534d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:55.573607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f90e5-75b6-4889-b453-02fbb1534d8c None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:55.574457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f90e5-75b6-4889-b453-02fbb1534d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:55.575011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f90e5-75b6-4889-b453-02fbb1534d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:55.576711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f90e5-75b6-4889-b453-02fbb1534d8c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:55.590437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 210/426] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:55.596089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ff4f5-6139-4073-b1af-74b93921d0ef None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:55.692509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750784ae-3be9-406e-9725-39402d4f957d None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:55.693023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750784ae-3be9-406e-9725-39402d4f957d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:55.693488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750784ae-3be9-406e-9725-39402d4f957d None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:55.695105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 211/427] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:53:55.706776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880ac25-82c7-4fc8-a4fb-9c76eb70e1f0 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:55.707506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880ac25-82c7-4fc8-a4fb-9c76eb70e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:55.707982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880ac25-82c7-4fc8-a4fb-9c76eb70e1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:55.729236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ff4f5-6139-4073-b1af-74b93921d0ef None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:55.748431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2880ac25-82c7-4fc8-a4fb-9c76eb70e1f0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:55.759038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2880ac25-82c7-4fc8-a4fb-9c76eb70e1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:55.759236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:55.804256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86e8a8-7c28-4b9b-bb24-ae160923e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:55.804895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86e8a8-7c28-4b9b-bb24-ae160923e399 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:55.805167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86e8a8-7c28-4b9b-bb24-ae160923e399 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:55.805431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86e8a8-7c28-4b9b-bb24-ae160923e399 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:55.822766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86e8a8-7c28-4b9b-bb24-ae160923e399 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:55.824272 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86e8a8-7c28-4b9b-bb24-ae160923e399 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-9f86e8a8-7c28-4b9b-bb24-ae160923e399 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:53:55.845818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:55.846713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:55.847988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 217/428] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:53:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:53:55.853360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86e8a8-7c28-4b9b-bb24-ae160923e399 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:55.887084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 212/429] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:53:55.888266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880ac25-82c7-4fc8-a4fb-9c76eb70e1f0 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:55.998021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:56.008868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86e8a8-7c28-4b9b-bb24-ae160923e399 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.058006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aa97e-bff7-46e6-a9a3-a39b7abc36e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.058609 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa97e-bff7-46e6-a9a3-a39b7abc36e2 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.058890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa97e-bff7-46e6-a9a3-a39b7abc36e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.059179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aa97e-bff7-46e6-a9a3-a39b7abc36e2 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.059927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aa97e-bff7-46e6-a9a3-a39b7abc36e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:56.060229 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aa97e-bff7-46e6-a9a3-a39b7abc36e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:56.061713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aa97e-bff7-46e6-a9a3-a39b7abc36e2 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:56.069068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880ac25-82c7-4fc8-a4fb-9c76eb70e1f0 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.079705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 218/430] 10.222.0.58 () {56 vars in 1135 bytes} [Fri Oct 28 18:53:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:56.080913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f629260-6a20-4015-912f-09cf5d77aaa8 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.081194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f629260-6a20-4015-912f-09cf5d77aaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.081494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f629260-6a20-4015-912f-09cf5d77aaa8 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.086809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 219/431] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:56.087592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f629260-6a20-4015-912f-09cf5d77aaa8 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.118660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:56.119290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b27dc-d8de-4033-9586-626e609aaf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.120039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b27dc-d8de-4033-9586-626e609aaf80 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.120676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b27dc-d8de-4033-9586-626e609aaf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.120900 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b27dc-d8de-4033-9586-626e609aaf80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.134146 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b27dc-d8de-4033-9586-626e609aaf80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-897b27dc-d8de-4033-9586-626e609aaf80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:56.139296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:56.140198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:56.141000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 213/432] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:56.144824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78435f-a543-4db3-8732-83035a6cb413 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.144824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78435f-a543-4db3-8732-83035a6cb413 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.144824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78435f-a543-4db3-8732-83035a6cb413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.161795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3c78435f-a543-4db3-8732-83035a6cb413 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:56.163117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3c78435f-a543-4db3-8732-83035a6cb413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:56.200617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f629260-6a20-4015-912f-09cf5d77aaa8 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.243008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 214/433] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:56.243008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78435f-a543-4db3-8732-83035a6cb413 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.245586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a0dbc-48df-4444-afc4-f835f24ebc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.246338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a0dbc-48df-4444-afc4-f835f24ebc38 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.246642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a0dbc-48df-4444-afc4-f835f24ebc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.247855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a0dbc-48df-4444-afc4-f835f24ebc38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.260479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a0dbc-48df-4444-afc4-f835f24ebc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:56.261921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a0dbc-48df-4444-afc4-f835f24ebc38 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:56.276716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-c72a0dbc-48df-4444-afc4-f835f24ebc38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:56.286636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:56.287567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:56.288394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:56.288394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:56.288394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:56.291109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 220/434] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:56.292239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a0dbc-48df-4444-afc4-f835f24ebc38 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.362629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78435f-a543-4db3-8732-83035a6cb413 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.386010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:56.407216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a0dbc-48df-4444-afc4-f835f24ebc38 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.411966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7783ad2-46df-4c75-9a51-d418806e73eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.412580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7783ad2-46df-4c75-9a51-d418806e73eb None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.412857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7783ad2-46df-4c75-9a51-d418806e73eb None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.413127 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7783ad2-46df-4c75-9a51-d418806e73eb None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.413785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7783ad2-46df-4c75-9a51-d418806e73eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:56.414081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7783ad2-46df-4c75-9a51-d418806e73eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:56.415484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7783ad2-46df-4c75-9a51-d418806e73eb None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:56.428351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 215/435] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:56.429371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7783ad2-46df-4c75-9a51-d418806e73eb None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.448970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb59f17-a5fb-4264-8fe8-9a0acb34681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.449893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59f17-a5fb-4264-8fe8-9a0acb34681c None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.450387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59f17-a5fb-4264-8fe8-9a0acb34681c None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.450851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59f17-a5fb-4264-8fe8-9a0acb34681c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.462702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb59f17-a5fb-4264-8fe8-9a0acb34681c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-cbb59f17-a5fb-4264-8fe8-9a0acb34681c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:56.468843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:56.469714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:56.470495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:56.477455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 221/436] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:56.478423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb59f17-a5fb-4264-8fe8-9a0acb34681c None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.542935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:56.547137 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7783ad2-46df-4c75-9a51-d418806e73eb None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.583614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bd0ba-bb9e-40f4-aace-d56b213ecc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.584208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bd0ba-bb9e-40f4-aace-d56b213ecc2e None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.584480 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bd0ba-bb9e-40f4-aace-d56b213ecc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.584738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bd0ba-bb9e-40f4-aace-d56b213ecc2e None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.585367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bd0ba-bb9e-40f4-aace-d56b213ecc2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:56.585663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bd0ba-bb9e-40f4-aace-d56b213ecc2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:56.587056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bd0ba-bb9e-40f4-aace-d56b213ecc2e None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:56.588686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb59f17-a5fb-4264-8fe8-9a0acb34681c None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.603034 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 216/437] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:56.603034 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bd0ba-bb9e-40f4-aace-d56b213ecc2e None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.628379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcdbd3-4c2b-4f94-a91d-49e6a0ee0ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.629003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcdbd3-4c2b-4f94-a91d-49e6a0ee0ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.629330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcdbd3-4c2b-4f94-a91d-49e6a0ee0ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.629648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcdbd3-4c2b-4f94-a91d-49e6a0ee0ff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.641160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcdbd3-4c2b-4f94-a91d-49e6a0ee0ff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:56.642595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcdbd3-4c2b-4f94-a91d-49e6a0ee0ff5 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:56.644820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-72bcdbd3-4c2b-4f94-a91d-49e6a0ee0ff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:56.658829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:56.659911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:56.660769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:56.660769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:56.660769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:56.660769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 222/438] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:56.660981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcdbd3-4c2b-4f94-a91d-49e6a0ee0ff5 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.716006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bd0ba-bb9e-40f4-aace-d56b213ecc2e None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.759326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6bd0e-b58f-4a2f-b858-972c1bed2815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.759932 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6bd0e-b58f-4a2f-b858-972c1bed2815 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.760209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6bd0e-b58f-4a2f-b858-972c1bed2815 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.760474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6bd0e-b58f-4a2f-b858-972c1bed2815 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.761127 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6bd0e-b58f-4a2f-b858-972c1bed2815 None admin] RBAC: Adding query filter params (name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:56.761425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6bd0e-b58f-4a2f-b858-972c1bed2815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:56.762947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6bd0e-b58f-4a2f-b858-972c1bed2815 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:56.770474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcdbd3-4c2b-4f94-a91d-49e6a0ee0ff5 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.776101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 217/439] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:56.776101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6bd0e-b58f-4a2f-b858-972c1bed2815 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.799836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:56.810528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ba5fd-dc13-4820-b226-e115b67f3bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.811185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ba5fd-dc13-4820-b226-e115b67f3bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.811455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ba5fd-dc13-4820-b226-e115b67f3bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.811722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ba5fd-dc13-4820-b226-e115b67f3bb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.821646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ba5fd-dc13-4820-b226-e115b67f3bb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-b58ba5fd-dc13-4820-b226-e115b67f3bb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:56.826889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:56.827852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:56.828665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 223/440] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:56.829318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ba5fd-dc13-4820-b226-e115b67f3bb2 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.876782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6bd0e-b58f-4a2f-b858-972c1bed2815 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.882804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:56.910198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b4fd5-0d10-43c3-ae25-596c88d32f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.910822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b4fd5-0d10-43c3-ae25-596c88d32f46 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.911138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b4fd5-0d10-43c3-ae25-596c88d32f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.911412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b4fd5-0d10-43c3-ae25-596c88d32f46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.917752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b4fd5-0d10-43c3-ae25-596c88d32f46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:56.919822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b4fd5-0d10-43c3-ae25-596c88d32f46 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:56.926860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ba5fd-dc13-4820-b226-e115b67f3bb2 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-5f8b4fd5-0d10-43c3-ae25-596c88d32f46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:53:56.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:56.931600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:56.932462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:53:56.932462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:53:56.932462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:53:56.932462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:53:56.932462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:53:56.932462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:53:56.932462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:56.932462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:56.932462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 218/441] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:56.932932 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b4fd5-0d10-43c3-ae25-596c88d32f46 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:56.959486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc650d14-c1d4-4319-a548-c563b9cc2333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:56.960079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc650d14-c1d4-4319-a548-c563b9cc2333 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.960355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc650d14-c1d4-4319-a548-c563b9cc2333 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.960620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc650d14-c1d4-4319-a548-c563b9cc2333 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.961258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc650d14-c1d4-4319-a548-c563b9cc2333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:56.961557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc650d14-c1d4-4319-a548-c563b9cc2333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:56.962909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc650d14-c1d4-4319-a548-c563b9cc2333 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:56.974680 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 224/442] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:56.975858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790120f-9070-411d-bf6b-2a3fe4faed60 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:56.976139 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790120f-9070-411d-bf6b-2a3fe4faed60 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:56.976407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790120f-9070-411d-bf6b-2a3fe4faed60 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:56.977758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 225/443] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:53:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:53:56.978269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790120f-9070-411d-bf6b-2a3fe4faed60 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.019449 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:57.037932 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b4fd5-0d10-43c3-ae25-596c88d32f46 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.071158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcab6e5-fb2f-4558-a00f-8c590d76e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.071787 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcab6e5-fb2f-4558-a00f-8c590d76e13a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.072065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcab6e5-fb2f-4558-a00f-8c590d76e13a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.072336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcab6e5-fb2f-4558-a00f-8c590d76e13a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.078232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790120f-9070-411d-bf6b-2a3fe4faed60 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.083194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcab6e5-fb2f-4558-a00f-8c590d76e13a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.084640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcab6e5-fb2f-4558-a00f-8c590d76e13a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.092027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-5dcab6e5-fb2f-4558-a00f-8c590d76e13a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:57.099443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:57.100353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:57.101209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:57.101209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:57.101209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:57.101209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 219/444] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:57.101434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcab6e5-fb2f-4558-a00f-8c590d76e13a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.117342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a1970-b36b-43c9-a761-8029b0b0c9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.117943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a1970-b36b-43c9-a761-8029b0b0c9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.118221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a1970-b36b-43c9-a761-8029b0b0c9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.118489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a1970-b36b-43c9-a761-8029b0b0c9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.119172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a1970-b36b-43c9-a761-8029b0b0c9b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:57.119473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a1970-b36b-43c9-a761-8029b0b0c9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.120837 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a1970-b36b-43c9-a761-8029b0b0c9b7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.128632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 226/445] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:53:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:57.129401 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a1970-b36b-43c9-a761-8029b0b0c9b7 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.197022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcab6e5-fb2f-4558-a00f-8c590d76e13a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.230314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d55c14-2c10-4948-a2d2-b92261246751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.230905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d55c14-2c10-4948-a2d2-b92261246751 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.231209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d55c14-2c10-4948-a2d2-b92261246751 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.231479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d55c14-2c10-4948-a2d2-b92261246751 None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.231705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a1970-b36b-43c9-a761-8029b0b0c9b7 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.232164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d55c14-2c10-4948-a2d2-b92261246751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:57.232465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d55c14-2c10-4948-a2d2-b92261246751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.233911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d55c14-2c10-4948-a2d2-b92261246751 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.235602 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:57.252753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 220/446] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:57.254022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142c426-91a6-4b34-838e-e30fc6f6ffa5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.254308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142c426-91a6-4b34-838e-e30fc6f6ffa5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.254581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142c426-91a6-4b34-838e-e30fc6f6ffa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.269526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c396-4a0e-45ad-ab0d-5276d60ba1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.270228 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c396-4a0e-45ad-ab0d-5276d60ba1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.270502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c396-4a0e-45ad-ab0d-5276d60ba1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.270766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c396-4a0e-45ad-ab0d-5276d60ba1d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.272536 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6142c426-91a6-4b34-838e-e30fc6f6ffa5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:53:57.273717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6142c426-91a6-4b34-838e-e30fc6f6ffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:53:57.278056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7c396-4a0e-45ad-ab0d-5276d60ba1d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.279984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7c396-4a0e-45ad-ab0d-5276d60ba1d7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-5cc7c396-4a0e-45ad-ab0d-5276d60ba1d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:53:57.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:57.292519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 227/447] 10.222.0.58 () {56 vars in 1093 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:57.298291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c396-4a0e-45ad-ab0d-5276d60ba1d7 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.327977 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:57.349566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 221/448] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:53:57.350403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142c426-91a6-4b34-838e-e30fc6f6ffa5 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.394575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c396-4a0e-45ad-ab0d-5276d60ba1d7 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.429190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a66e11-1233-423e-9137-4b3bdbcd0a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.429790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a66e11-1233-423e-9137-4b3bdbcd0a62 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.430091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a66e11-1233-423e-9137-4b3bdbcd0a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.430343 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a66e11-1233-423e-9137-4b3bdbcd0a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.430974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a66e11-1233-423e-9137-4b3bdbcd0a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:57.431299 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a66e11-1233-423e-9137-4b3bdbcd0a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.432655 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a66e11-1233-423e-9137-4b3bdbcd0a62 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.444941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 228/449] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:57.445533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a66e11-1233-423e-9137-4b3bdbcd0a62 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.473622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142c426-91a6-4b34-838e-e30fc6f6ffa5 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.498671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:57.511461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba79616-d7c2-428e-a2ac-2d31944b648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.512422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba79616-d7c2-428e-a2ac-2d31944b648c None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.513403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba79616-d7c2-428e-a2ac-2d31944b648c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.513861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba79616-d7c2-428e-a2ac-2d31944b648c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.524657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba79616-d7c2-428e-a2ac-2d31944b648c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.526306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba79616-d7c2-428e-a2ac-2d31944b648c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.539498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-1ba79616-d7c2-428e-a2ac-2d31944b648c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:57.544180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:57.545050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:57.545896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:57.545896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:57.545896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:57.546055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a66e11-1233-423e-9137-4b3bdbcd0a62 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.547785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 222/450] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:57.548694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba79616-d7c2-428e-a2ac-2d31944b648c None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.586260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a027bf-540a-49c1-847d-533a2e045a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.586915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a027bf-540a-49c1-847d-533a2e045a71 None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.587218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a027bf-540a-49c1-847d-533a2e045a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.587491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a027bf-540a-49c1-847d-533a2e045a71 None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a51690d0367b45db8a6552764cc1d11e/roles/01b71dd1f22e41269c3caa83dea57b2e` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.602030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a027bf-540a-49c1-847d-533a2e045a71 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=a51690d0367b45db8a6552764cc1d11e, role_id=01b71dd1f22e41269c3caa83dea57b2e)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.603501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a027bf-540a-49c1-847d-533a2e045a71 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.617768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 229/451] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:53:57 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a51690d0367b45db8a6552764cc1d11e/roles/01b71dd1f22e41269c3caa83dea57b2e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:53:57.618567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a027bf-540a-49c1-847d-533a2e045a71 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.663225 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba79616-d7c2-428e-a2ac-2d31944b648c None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.701371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79c36e-1f6f-4c95-9ecd-8917f2c230fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.701986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79c36e-1f6f-4c95-9ecd-8917f2c230fe None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.702265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79c36e-1f6f-4c95-9ecd-8917f2c230fe None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.702593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79c36e-1f6f-4c95-9ecd-8917f2c230fe None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.703381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79c36e-1f6f-4c95-9ecd-8917f2c230fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:57.703689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79c36e-1f6f-4c95-9ecd-8917f2c230fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.705158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79c36e-1f6f-4c95-9ecd-8917f2c230fe None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.724809 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 223/452] 10.222.0.58 () {56 vars in 1133 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:57.725593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79c36e-1f6f-4c95-9ecd-8917f2c230fe None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.747913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a027bf-540a-49c1-847d-533a2e045a71 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.768058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:57.793179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38654834-a42e-46c4-ad74-708f44ce9b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.793868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38654834-a42e-46c4-ad74-708f44ce9b84 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.794174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38654834-a42e-46c4-ad74-708f44ce9b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.794444 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38654834-a42e-46c4-ad74-708f44ce9b84 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.802547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38654834-a42e-46c4-ad74-708f44ce9b84 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.804047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38654834-a42e-46c4-ad74-708f44ce9b84 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-38654834-a42e-46c4-ad74-708f44ce9b84 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:53:57.816295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:57.817367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 28 18:53:57.818440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:57.820229 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 230/453] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:57.821005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38654834-a42e-46c4-ad74-708f44ce9b84 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.832921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:57.846026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79c36e-1f6f-4c95-9ecd-8917f2c230fe None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.883887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8479d59-96e7-4c9a-9496-784af1347fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.884501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8479d59-96e7-4c9a-9496-784af1347fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.884810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8479d59-96e7-4c9a-9496-784af1347fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.885047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8479d59-96e7-4c9a-9496-784af1347fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.885689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8479d59-96e7-4c9a-9496-784af1347fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:57.885988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8479d59-96e7-4c9a-9496-784af1347fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:57.887532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8479d59-96e7-4c9a-9496-784af1347fa3 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:57.899488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 224/454] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:57.900134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8479d59-96e7-4c9a-9496-784af1347fa3 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.936418 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38654834-a42e-46c4-ad74-708f44ce9b84 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:57.970859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6a2ec-eebd-438a-8143-1a86c9e64dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:57.971757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6a2ec-eebd-438a-8143-1a86c9e64dbf None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:57.972279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6a2ec-eebd-438a-8143-1a86c9e64dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:57.972546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6a2ec-eebd-438a-8143-1a86c9e64dbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:57.983883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6a2ec-eebd-438a-8143-1a86c9e64dbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-fbc6a2ec-eebd-438a-8143-1a86c9e64dbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:53:57.989089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:57.989983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:53:57.990761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:57.994906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 231/455] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:57.995721 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6a2ec-eebd-438a-8143-1a86c9e64dbf None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:57.998627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:58.006548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8479d59-96e7-4c9a-9496-784af1347fa3 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:58.040575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59ddb5-3aad-4b9e-815c-28e0978c4403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:58.041212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59ddb5-3aad-4b9e-815c-28e0978c4403 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:58.041438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59ddb5-3aad-4b9e-815c-28e0978c4403 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:58.041710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59ddb5-3aad-4b9e-815c-28e0978c4403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:58.067067 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:58.106890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6a2ec-eebd-438a-8143-1a86c9e64dbf None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:58.114335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59ddb5-3aad-4b9e-815c-28e0978c4403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:58.116238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59ddb5-3aad-4b9e-815c-28e0978c4403 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:58.144096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d82777-0c7d-4047-b2f3-0424eb721583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:58.144694 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d82777-0c7d-4047-b2f3-0424eb721583 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:58.144971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d82777-0c7d-4047-b2f3-0424eb721583 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:58.145239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d82777-0c7d-4047-b2f3-0424eb721583 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:58.145956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d82777-0c7d-4047-b2f3-0424eb721583 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:58.146256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d82777-0c7d-4047-b2f3-0424eb721583 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:58.147706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d82777-0c7d-4047-b2f3-0424eb721583 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:58.169588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 232/456] 10.222.0.58 () {56 vars in 1137 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:58.169588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d82777-0c7d-4047-b2f3-0424eb721583 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:58.219121 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:58.228033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 225/457] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:58.233038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59ddb5-3aad-4b9e-815c-28e0978c4403 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:58.273617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d82777-0c7d-4047-b2f3-0424eb721583 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:58.299524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:58.310814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4aa592-3f2e-4d09-9806-ae768c85eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:58.311555 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4aa592-3f2e-4d09-9806-ae768c85eb5c None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:58.311833 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4aa592-3f2e-4d09-9806-ae768c85eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:58.312103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4aa592-3f2e-4d09-9806-ae768c85eb5c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:58.319347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4aa592-3f2e-4d09-9806-ae768c85eb5c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:58.320788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4aa592-3f2e-4d09-9806-ae768c85eb5c None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-dc4aa592-3f2e-4d09-9806-ae768c85eb5c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:53:58.333164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:58.334084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 233/458] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:53:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:58.335025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4aa592-3f2e-4d09-9806-ae768c85eb5c None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:58.344710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59ddb5-3aad-4b9e-815c-28e0978c4403 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:58.385085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d5853-6738-45cb-9757-32c8dfe789e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:58.385826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d5853-6738-45cb-9757-32c8dfe789e7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:58.386104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d5853-6738-45cb-9757-32c8dfe789e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:58.386370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d5853-6738-45cb-9757-32c8dfe789e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:58.387053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d5853-6738-45cb-9757-32c8dfe789e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:58.387379 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d5853-6738-45cb-9757-32c8dfe789e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:58.388743 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d5853-6738-45cb-9757-32c8dfe789e7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:58.397042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 226/459] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:53:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:58.402345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d5853-6738-45cb-9757-32c8dfe789e7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:58.443964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4aa592-3f2e-4d09-9806-ae768c85eb5c None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:58.461271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:58.480957 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343e715-b33e-4a84-a45a-6f8ecfb8aa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:58.481597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343e715-b33e-4a84-a45a-6f8ecfb8aa24 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:58.481868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343e715-b33e-4a84-a45a-6f8ecfb8aa24 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:58.482130 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343e715-b33e-4a84-a45a-6f8ecfb8aa24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:58.510330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d5853-6738-45cb-9757-32c8dfe789e7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:58.542020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:58.545964 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf176a49-b6ce-492c-b5ab-1c7f62bc9279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:58.546725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf176a49-b6ce-492c-b5ab-1c7f62bc9279 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:58.547048 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf176a49-b6ce-492c-b5ab-1c7f62bc9279 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:58.547345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf176a49-b6ce-492c-b5ab-1c7f62bc9279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:58.557960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf176a49-b6ce-492c-b5ab-1c7f62bc9279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:58.559990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf176a49-b6ce-492c-b5ab-1c7f62bc9279 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:58.560099 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343e715-b33e-4a84-a45a-6f8ecfb8aa24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:58.561412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343e715-b33e-4a84-a45a-6f8ecfb8aa24 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-bf176a49-b6ce-492c-b5ab-1c7f62bc9279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:58.574031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:58.576247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:58.577515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:58.577515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:58.577515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:58.577515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 227/460] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:58.580580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf176a49-b6ce-492c-b5ab-1c7f62bc9279 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:58.668457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 234/461] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:58.670415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343e715-b33e-4a84-a45a-6f8ecfb8aa24 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:58.683675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf176a49-b6ce-492c-b5ab-1c7f62bc9279 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:58.693567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:58.720247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4459c2-8c9b-4972-95cf-4053f23188cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:58.720843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4459c2-8c9b-4972-95cf-4053f23188cd None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:58.721127 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4459c2-8c9b-4972-95cf-4053f23188cd None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:58.721401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4459c2-8c9b-4972-95cf-4053f23188cd None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:58.722124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4459c2-8c9b-4972-95cf-4053f23188cd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:58.722438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4459c2-8c9b-4972-95cf-4053f23188cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:58.723903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4459c2-8c9b-4972-95cf-4053f23188cd None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:58.747626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 228/462] 10.222.0.58 () {56 vars in 1137 bytes} [Fri Oct 28 18:53:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:58.747626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4459c2-8c9b-4972-95cf-4053f23188cd None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:58.772999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:58.790610 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343e715-b33e-4a84-a45a-6f8ecfb8aa24 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:58.829540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c77840-c67b-4a7a-a924-82850db3d50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:58.830261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c77840-c67b-4a7a-a924-82850db3d50b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:58.830365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c77840-c67b-4a7a-a924-82850db3d50b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:58.830635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c77840-c67b-4a7a-a924-82850db3d50b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:58.841455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c77840-c67b-4a7a-a924-82850db3d50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:58.842910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c77840-c67b-4a7a-a924-82850db3d50b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:58.847886 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4459c2-8c9b-4972-95cf-4053f23188cd None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-61c77840-c67b-4a7a-a924-82850db3d50b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:58.861158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:58.862081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:58.867885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:58.867885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:58.867885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:58.867885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 235/463] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:58.867885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c77840-c67b-4a7a-a924-82850db3d50b None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:58.886456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa72ee5-ccb9-42bd-b7f8-0d38f0671cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:58.887185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa72ee5-ccb9-42bd-b7f8-0d38f0671cae None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:58.887494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa72ee5-ccb9-42bd-b7f8-0d38f0671cae None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:58.887762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa72ee5-ccb9-42bd-b7f8-0d38f0671cae None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:58.888396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa72ee5-ccb9-42bd-b7f8-0d38f0671cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:58.888695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa72ee5-ccb9-42bd-b7f8-0d38f0671cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:58.890058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa72ee5-ccb9-42bd-b7f8-0d38f0671cae None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:58.901356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 229/464] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:58.902143 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa72ee5-ccb9-42bd-b7f8-0d38f0671cae None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:58.934152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:58.983485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c77840-c67b-4a7a-a924-82850db3d50b None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.012025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa72ee5-ccb9-42bd-b7f8-0d38f0671cae None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.018834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d7682-c23e-4f80-9dca-32f8e20847b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.019484 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d7682-c23e-4f80-9dca-32f8e20847b3 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.019764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d7682-c23e-4f80-9dca-32f8e20847b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.020030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d7682-c23e-4f80-9dca-32f8e20847b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.023004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:59.032716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d7682-c23e-4f80-9dca-32f8e20847b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.034189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d7682-c23e-4f80-9dca-32f8e20847b3 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.048136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14bfa0-770e-4f5c-8429-2689fe3f32b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.048747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14bfa0-770e-4f5c-8429-2689fe3f32b7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.049001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14bfa0-770e-4f5c-8429-2689fe3f32b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.049269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14bfa0-770e-4f5c-8429-2689fe3f32b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-f78d7682-c23e-4f80-9dca-32f8e20847b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:59.049363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:59.050207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:59.051081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:59.051081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:59.051081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:59.055136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 236/465] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:59.055960 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d7682-c23e-4f80-9dca-32f8e20847b3 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.124253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14bfa0-770e-4f5c-8429-2689fe3f32b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.125777 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14bfa0-770e-4f5c-8429-2689fe3f32b7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.152214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:59.161652 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d7682-c23e-4f80-9dca-32f8e20847b3 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.195614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077d2d0-08b6-4769-9fac-0c2f68e4361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.196220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077d2d0-08b6-4769-9fac-0c2f68e4361c None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.196502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077d2d0-08b6-4769-9fac-0c2f68e4361c None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.196770 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077d2d0-08b6-4769-9fac-0c2f68e4361c None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.197403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077d2d0-08b6-4769-9fac-0c2f68e4361c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:59.197706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077d2d0-08b6-4769-9fac-0c2f68e4361c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.199111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077d2d0-08b6-4769-9fac-0c2f68e4361c None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.210794 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 237/466] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:59.211621 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077d2d0-08b6-4769-9fac-0c2f68e4361c None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.234667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 230/467] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:53:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:53:59.235763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14bfa0-770e-4f5c-8429-2689fe3f32b7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.273718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:59.329774 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077d2d0-08b6-4769-9fac-0c2f68e4361c None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.345589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14bfa0-770e-4f5c-8429-2689fe3f32b7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.364940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bbc8a-061e-4061-802f-51ea116e6c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.365571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bbc8a-061e-4061-802f-51ea116e6c6f None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.365964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bbc8a-061e-4061-802f-51ea116e6c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.366236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bbc8a-061e-4061-802f-51ea116e6c6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.373761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bbc8a-061e-4061-802f-51ea116e6c6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.375583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bbc8a-061e-4061-802f-51ea116e6c6f None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.376945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:59.386244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57707fee-ea51-4328-a443-ca2938d87878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.386847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57707fee-ea51-4328-a443-ca2938d87878 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.387158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57707fee-ea51-4328-a443-ca2938d87878 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.387969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57707fee-ea51-4328-a443-ca2938d87878 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.388607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57707fee-ea51-4328-a443-ca2938d87878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-a24bbc8a-061e-4061-802f-51ea116e6c6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:53:59.388701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:59.389472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:59.395537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:53:59.395537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:53:59.395537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:53:59.395537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:53:59.395537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:53:59.395537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:53:59.395537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:59.395537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:59.395945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57707fee-ea51-4328-a443-ca2938d87878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.396014 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 238/468] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:59.396094 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57707fee-ea51-4328-a443-ca2938d87878 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.396162 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bbc8a-061e-4061-802f-51ea116e6c6f None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.411655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 231/469] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:59.411655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57707fee-ea51-4328-a443-ca2938d87878 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.504866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bbc8a-061e-4061-802f-51ea116e6c6f None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.513966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57707fee-ea51-4328-a443-ca2938d87878 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.516915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:59.544563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f4cf1-a368-4bd0-93a4-95982e8097f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.545190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f4cf1-a368-4bd0-93a4-95982e8097f7 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.545473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f4cf1-a368-4bd0-93a4-95982e8097f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.545744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f4cf1-a368-4bd0-93a4-95982e8097f7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.551924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2784f7-3522-4383-8b54-66f9c6829228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.552985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2784f7-3522-4383-8b54-66f9c6829228 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.553211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f4cf1-a368-4bd0-93a4-95982e8097f7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.553573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2784f7-3522-4383-8b54-66f9c6829228 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.553999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2784f7-3522-4383-8b54-66f9c6829228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.554635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f4cf1-a368-4bd0-93a4-95982e8097f7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-8f8f4cf1-a368-4bd0-93a4-95982e8097f7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:53:59.566114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:59.567234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 239/470] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:59.568269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f4cf1-a368-4bd0-93a4-95982e8097f7 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.569212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2784f7-3522-4383-8b54-66f9c6829228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.570829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2784f7-3522-4383-8b54-66f9c6829228 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-4b2784f7-3522-4383-8b54-66f9c6829228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:53:59.589544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:59.591040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:53:59.591885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:53:59.591885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:53:59.591885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:59.591885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 232/471] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:59.592680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2784f7-3522-4383-8b54-66f9c6829228 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.643925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:59.676119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f4cf1-a368-4bd0-93a4-95982e8097f7 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.713345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2784f7-3522-4383-8b54-66f9c6829228 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.716704 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b430ad1-b56a-4f70-9bc3-9719118f0934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.717463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b430ad1-b56a-4f70-9bc3-9719118f0934 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.717739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b430ad1-b56a-4f70-9bc3-9719118f0934 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.718013 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b430ad1-b56a-4f70-9bc3-9719118f0934 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.718699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b430ad1-b56a-4f70-9bc3-9719118f0934 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:59.719084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b430ad1-b56a-4f70-9bc3-9719118f0934 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.720567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b430ad1-b56a-4f70-9bc3-9719118f0934 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.740449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 240/472] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:59.740449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b430ad1-b56a-4f70-9bc3-9719118f0934 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.759271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d92f76-f902-4fc2-888e-9e74be747447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.759977 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d92f76-f902-4fc2-888e-9e74be747447 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.760326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d92f76-f902-4fc2-888e-9e74be747447 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.760695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d92f76-f902-4fc2-888e-9e74be747447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.769073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d92f76-f902-4fc2-888e-9e74be747447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.770563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d92f76-f902-4fc2-888e-9e74be747447 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.771955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-71d92f76-f902-4fc2-888e-9e74be747447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:53:59.782433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:53:59.783315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:53:59.784125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:53:59.784125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:53:59.784125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:53:59.784125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:53:59.784125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:53:59.784125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:53:59.784125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:53:59.784125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:53:59.784125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 233/473] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:53:59.785063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d92f76-f902-4fc2-888e-9e74be747447 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.848349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b430ad1-b56a-4f70-9bc3-9719118f0934 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.879840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:53:59.880818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981764f-1d65-40ca-9b7a-6b62984438b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.881423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981764f-1d65-40ca-9b7a-6b62984438b0 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.881698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981764f-1d65-40ca-9b7a-6b62984438b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.881967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981764f-1d65-40ca-9b7a-6b62984438b0 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.882674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981764f-1d65-40ca-9b7a-6b62984438b0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:59.883008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981764f-1d65-40ca-9b7a-6b62984438b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.884067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d92f76-f902-4fc2-888e-9e74be747447 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:53:59.884427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981764f-1d65-40ca-9b7a-6b62984438b0 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.901801 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 241/474] 10.222.0.58 () {56 vars in 1139 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:59.905668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981764f-1d65-40ca-9b7a-6b62984438b0 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.917739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66a44e-a5ba-40a8-8b98-75206bbab9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:53:59.918334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66a44e-a5ba-40a8-8b98-75206bbab9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:53:59.918687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66a44e-a5ba-40a8-8b98-75206bbab9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:53:59.918955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66a44e-a5ba-40a8-8b98-75206bbab9f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:53:59.919609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66a44e-a5ba-40a8-8b98-75206bbab9f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:53:59.919907 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66a44e-a5ba-40a8-8b98-75206bbab9f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:53:59.921268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66a44e-a5ba-40a8-8b98-75206bbab9f2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:53:59.932472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 234/475] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:53:59.933238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66a44e-a5ba-40a8-8b98-75206bbab9f2 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:53:59.986847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:00.002628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981764f-1d65-40ca-9b7a-6b62984438b0 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:00.031278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66a44e-a5ba-40a8-8b98-75206bbab9f2 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:00.036280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8baacf-9dc7-4891-8809-a398768e9282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:00.036867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8baacf-9dc7-4891-8809-a398768e9282 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.037141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8baacf-9dc7-4891-8809-a398768e9282 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.037407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8baacf-9dc7-4891-8809-a398768e9282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.038110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8baacf-9dc7-4891-8809-a398768e9282 None admin] RBAC: Adding query filter params (role.id=01b71dd1f22e41269c3caa83dea57b2e, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=a758611cf8d94e8ca8aad6a75bef5e34) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:00.038412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8baacf-9dc7-4891-8809-a398768e9282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:00.039855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8baacf-9dc7-4891-8809-a398768e9282 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:00.059009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 242/476] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/role_assignments?user.id=a758611cf8d94e8ca8aad6a75bef5e34&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=01b71dd1f22e41269c3caa83dea57b2e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:00.059009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d34cf5-f53c-487e-84cf-1bda1d8a03e7 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.059009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d34cf5-f53c-487e-84cf-1bda1d8a03e7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.059009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d34cf5-f53c-487e-84cf-1bda1d8a03e7 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.059009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 243/477] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:00.059009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d34cf5-f53c-487e-84cf-1bda1d8a03e7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:00.066890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962171b-f45a-4940-85ba-55307a721769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:00.067636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962171b-f45a-4940-85ba-55307a721769 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.067915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962171b-f45a-4940-85ba-55307a721769 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.068177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962171b-f45a-4940-85ba-55307a721769 None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.068878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962171b-f45a-4940-85ba-55307a721769 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:00.069182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962171b-f45a-4940-85ba-55307a721769 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:00.070661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962171b-f45a-4940-85ba-55307a721769 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:00.089533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 235/478] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:53:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:00.089533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962171b-f45a-4940-85ba-55307a721769 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:00.110223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:00.173239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d34cf5-f53c-487e-84cf-1bda1d8a03e7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:00.204616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962171b-f45a-4940-85ba-55307a721769 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:00.211095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a03894-8c5d-41ac-8df0-be14212a3311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:00.211719 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a03894-8c5d-41ac-8df0-be14212a3311 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.211980 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a03894-8c5d-41ac-8df0-be14212a3311 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.212247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a03894-8c5d-41ac-8df0-be14212a3311 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.235897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:00.245461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b9599-b2d3-45f8-bb24-5d2cae411963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:00.246095 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b9599-b2d3-45f8-bb24-5d2cae411963 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.246370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b9599-b2d3-45f8-bb24-5d2cae411963 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.247319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b9599-b2d3-45f8-bb24-5d2cae411963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.255238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b9599-b2d3-45f8-bb24-5d2cae411963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:00.256632 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b9599-b2d3-45f8-bb24-5d2cae411963 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-863b9599-b2d3-45f8-bb24-5d2cae411963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:00.270040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:00.270920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:00.271767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:00.271767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:00.271767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:00.271767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:00.271767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:00.271767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:00.271767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:00.271767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:00.271767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 236/479] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:00.272694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4993541-d2b7-4b66-9e40-a5511dbd85b2 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.272694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4993541-d2b7-4b66-9e40-a5511dbd85b2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.272974 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4993541-d2b7-4b66-9e40-a5511dbd85b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.290442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d4993541-d2b7-4b66-9e40-a5511dbd85b2 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:00.291006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a03894-8c5d-41ac-8df0-be14212a3311 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:00.291642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d4993541-d2b7-4b66-9e40-a5511dbd85b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:00.292476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a03894-8c5d-41ac-8df0-be14212a3311 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:00.360526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 237/480] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:54:00.361372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4993541-d2b7-4b66-9e40-a5511dbd85b2 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:00.374801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:00.423597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 244/481] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:54:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:54:00.424264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a03894-8c5d-41ac-8df0-be14212a3311 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:00.491392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:00.492698 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4993541-d2b7-4b66-9e40-a5511dbd85b2 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:00.526056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a03894-8c5d-41ac-8df0-be14212a3311 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:00.541212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67249a7-4310-40ce-8475-35535f06ce2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:00.542343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67249a7-4310-40ce-8475-35535f06ce2e None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.542798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67249a7-4310-40ce-8475-35535f06ce2e None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.543422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67249a7-4310-40ce-8475-35535f06ce2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.546702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67249a7-4310-40ce-8475-35535f06ce2e None admin] RBAC: Adding query filter params (role.id=01b71dd1f22e41269c3caa83dea57b2e, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=6e8ec16bd9254b23bfd2b60551af9cad) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:00.547239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67249a7-4310-40ce-8475-35535f06ce2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:00.548860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67249a7-4310-40ce-8475-35535f06ce2e None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:00.557790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 238/482] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:00 2022] GET /identity/v3/role_assignments?user.id=6e8ec16bd9254b23bfd2b60551af9cad&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=01b71dd1f22e41269c3caa83dea57b2e => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:00.559155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67249a7-4310-40ce-8475-35535f06ce2e None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:00.563923 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a35474-ae68-4817-9f8e-f47287171122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:00.564204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a35474-ae68-4817-9f8e-f47287171122 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.564693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a35474-ae68-4817-9f8e-f47287171122 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.565042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a35474-ae68-4817-9f8e-f47287171122 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.566025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a35474-ae68-4817-9f8e-f47287171122 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:00.566402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a35474-ae68-4817-9f8e-f47287171122 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:00.568055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a35474-ae68-4817-9f8e-f47287171122 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:00.582342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 245/483] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:00.589239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a35474-ae68-4817-9f8e-f47287171122 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:00.679595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:00.713906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67249a7-4310-40ce-8475-35535f06ce2e None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:00.747359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a35474-ae68-4817-9f8e-f47287171122 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:00.760767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:00.772250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd7a00-1a1c-4838-bca1-ee2ecde1406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:00.773712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd7a00-1a1c-4838-bca1-ee2ecde1406b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.773997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd7a00-1a1c-4838-bca1-ee2ecde1406b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.774274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd7a00-1a1c-4838-bca1-ee2ecde1406b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.789685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beeaba-0e61-409a-8a16-3f9781aa9689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:00.790354 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beeaba-0e61-409a-8a16-3f9781aa9689 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:00.790560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beeaba-0e61-409a-8a16-3f9781aa9689 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:00.791128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beeaba-0e61-409a-8a16-3f9781aa9689 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:00.791229 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd7a00-1a1c-4838-bca1-ee2ecde1406b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:00.791860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beeaba-0e61-409a-8a16-3f9781aa9689 None admin] RBAC: Adding query filter params (role.id=01b71dd1f22e41269c3caa83dea57b2e, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=302ded4fdd72405c8ce8a364f7d2400c) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:00.792163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beeaba-0e61-409a-8a16-3f9781aa9689 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:00.792695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd7a00-1a1c-4838-bca1-ee2ecde1406b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:00.794144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beeaba-0e61-409a-8a16-3f9781aa9689 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:00.801091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 246/484] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:00 2022] GET /identity/v3/role_assignments?user.id=302ded4fdd72405c8ce8a364f7d2400c&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=01b71dd1f22e41269c3caa83dea57b2e => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:00.802119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beeaba-0e61-409a-8a16-3f9781aa9689 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-d6bd7a00-1a1c-4838-bca1-ee2ecde1406b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:00.814405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:00.815392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:00.818099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:00.818099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:00.818099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:00.818099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 239/485] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:00.821815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd7a00-1a1c-4838-bca1-ee2ecde1406b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:00.956579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beeaba-0e61-409a-8a16-3f9781aa9689 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:00.979459 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd7a00-1a1c-4838-bca1-ee2ecde1406b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:01.029028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:01.031858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd55730-ef57-43d9-be39-3c1936bf1acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:01.032499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd55730-ef57-43d9-be39-3c1936bf1acd None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:01.032992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd55730-ef57-43d9-be39-3c1936bf1acd None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:01.033086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd55730-ef57-43d9-be39-3c1936bf1acd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:01.044416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf7397-7332-4f1a-86e0-8f86b15394cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:01.045022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf7397-7332-4f1a-86e0-8f86b15394cc None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:01.045292 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf7397-7332-4f1a-86e0-8f86b15394cc None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:01.045554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf7397-7332-4f1a-86e0-8f86b15394cc None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:01.046185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf7397-7332-4f1a-86e0-8f86b15394cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:01.046482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf7397-7332-4f1a-86e0-8f86b15394cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:01.048167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf7397-7332-4f1a-86e0-8f86b15394cc None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:01.052362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd55730-ef57-43d9-be39-3c1936bf1acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-4cd55730-ef57-43d9-be39-3c1936bf1acd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:01.059388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:01.060847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:54:01.061765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:01.064692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 247/486] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:54:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:01.066450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 240/487] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:01.070490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd55730-ef57-43d9-be39-3c1936bf1acd None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:01.074136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf7397-7332-4f1a-86e0-8f86b15394cc None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:01.113612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:01.204859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd55730-ef57-43d9-be39-3c1936bf1acd None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:01.220958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf7397-7332-4f1a-86e0-8f86b15394cc None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:01.266137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7f8f8-819a-4160-80c1-15277aa99c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:01.266792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7f8f8-819a-4160-80c1-15277aa99c60 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:01.267101 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7f8f8-819a-4160-80c1-15277aa99c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:01.267374 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7f8f8-819a-4160-80c1-15277aa99c60 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:01.268037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7f8f8-819a-4160-80c1-15277aa99c60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:01.268579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7f8f8-819a-4160-80c1-15277aa99c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:01.269971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7f8f8-819a-4160-80c1-15277aa99c60 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:01.280113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 248/488] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:54:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:01.283655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a27e5-e79e-49a3-a5a9-7b8b73daab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:01.284291 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a27e5-e79e-49a3-a5a9-7b8b73daab08 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:01.284568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a27e5-e79e-49a3-a5a9-7b8b73daab08 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:01.284835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a27e5-e79e-49a3-a5a9-7b8b73daab08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:01.286231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7f8f8-819a-4160-80c1-15277aa99c60 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:01.300947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a27e5-e79e-49a3-a5a9-7b8b73daab08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:01.302405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a27e5-e79e-49a3-a5a9-7b8b73daab08 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-7f4a27e5-e79e-49a3-a5a9-7b8b73daab08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:01.328792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:01.329699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:01.330525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:01.330525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:01.330525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:01.330525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:01.330525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:01.330525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:01.330525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:01.330525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:01.332212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 241/489] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:01.336981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a27e5-e79e-49a3-a5a9-7b8b73daab08 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:01.424512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:01.463432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7f8f8-819a-4160-80c1-15277aa99c60 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:01.467292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:01.505488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a27e5-e79e-49a3-a5a9-7b8b73daab08 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:01.535196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c82d70-9681-43ea-84cf-b58f0c1342da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:01.536883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c82d70-9681-43ea-84cf-b58f0c1342da None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:01.538532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c82d70-9681-43ea-84cf-b58f0c1342da None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:01.539929 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c82d70-9681-43ea-84cf-b58f0c1342da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:01.561238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c82d70-9681-43ea-84cf-b58f0c1342da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:01.562724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bafbf-98c5-4884-8878-2d685a42efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:01.563602 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bafbf-98c5-4884-8878-2d685a42efbf None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:01.564081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bafbf-98c5-4884-8878-2d685a42efbf None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:01.564549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bafbf-98c5-4884-8878-2d685a42efbf None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:01.565434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bafbf-98c5-4884-8878-2d685a42efbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:01.567114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c82d70-9681-43ea-84cf-b58f0c1342da None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:01.567648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bafbf-98c5-4884-8878-2d685a42efbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:01.569259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bafbf-98c5-4884-8878-2d685a42efbf None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-d8c82d70-9681-43ea-84cf-b58f0c1342da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:01.593755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:01.594693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:01.595575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:01.595575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:01.595575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:01.599880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 242/490] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:01.601463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 249/491] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:01.610466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bafbf-98c5-4884-8878-2d685a42efbf None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:01.611716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c82d70-9681-43ea-84cf-b58f0c1342da None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:01.761008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bafbf-98c5-4884-8878-2d685a42efbf None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:01.763580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:01.801231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4ff48-43a8-4f85-bdc8-b45b1cece6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:01.801840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4ff48-43a8-4f85-bdc8-b45b1cece6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:01.802119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4ff48-43a8-4f85-bdc8-b45b1cece6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:01.802391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4ff48-43a8-4f85-bdc8-b45b1cece6f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:01.803055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4ff48-43a8-4f85-bdc8-b45b1cece6f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:01.803364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4ff48-43a8-4f85-bdc8-b45b1cece6f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:01.804758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4ff48-43a8-4f85-bdc8-b45b1cece6f1 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:01.820159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c82d70-9681-43ea-84cf-b58f0c1342da None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:01.831746 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 243/492] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:01.837641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4ff48-43a8-4f85-bdc8-b45b1cece6f1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:01.878157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89110293-e2ce-400c-a824-e147e082dde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:01.878862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89110293-e2ce-400c-a824-e147e082dde6 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:01.879400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89110293-e2ce-400c-a824-e147e082dde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:01.879860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89110293-e2ce-400c-a824-e147e082dde6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:01.881074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89110293-e2ce-400c-a824-e147e082dde6 None admin] RBAC: Adding query filter params (role.id=01b71dd1f22e41269c3caa83dea57b2e, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=9cffb662f4104e349c81c578bf66ab7b) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:01.881377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89110293-e2ce-400c-a824-e147e082dde6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:01.887002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89110293-e2ce-400c-a824-e147e082dde6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:01.890465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 250/493] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:01 2022] GET /identity/v3/role_assignments?user.id=9cffb662f4104e349c81c578bf66ab7b&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=01b71dd1f22e41269c3caa83dea57b2e => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:01.984766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4ff48-43a8-4f85-bdc8-b45b1cece6f1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:02.038368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0667928-dd27-4220-b38d-6dbaaa82fdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:02.039418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0667928-dd27-4220-b38d-6dbaaa82fdcf None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:02.039944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0667928-dd27-4220-b38d-6dbaaa82fdcf None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:02.040433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0667928-dd27-4220-b38d-6dbaaa82fdcf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:02.050309 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0667928-dd27-4220-b38d-6dbaaa82fdcf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:02.052314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0667928-dd27-4220-b38d-6dbaaa82fdcf None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-c0667928-dd27-4220-b38d-6dbaaa82fdcf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:02.065334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:02.066332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 18:54:02.067278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:02.074486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 244/494] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:54:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:02.076780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89110293-e2ce-400c-a824-e147e082dde6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:02.101488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:02.185235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89110293-e2ce-400c-a824-e147e082dde6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:02.219451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d66a0-c402-4b51-8613-25597f321120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:02.220051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d66a0-c402-4b51-8613-25597f321120 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:02.220370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d66a0-c402-4b51-8613-25597f321120 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:02.220640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d66a0-c402-4b51-8613-25597f321120 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:02.221357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d66a0-c402-4b51-8613-25597f321120 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:02.221654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d66a0-c402-4b51-8613-25597f321120 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:02.223099 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d66a0-c402-4b51-8613-25597f321120 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:02.243561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 251/495] 10.222.0.58 () {56 vars in 1135 bytes} [Fri Oct 28 18:54:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:02.248346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0667928-dd27-4220-b38d-6dbaaa82fdcf None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:02.284343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:02.388103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0667928-dd27-4220-b38d-6dbaaa82fdcf None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:02.395511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e329a-7ec8-404d-abcc-7b263e8d8363 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:02.395763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e329a-7ec8-404d-abcc-7b263e8d8363 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:02.399003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e329a-7ec8-404d-abcc-7b263e8d8363 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:02.402419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 252/496] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:02.412268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54329d14-7146-402e-a48d-8063bb7f5932 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:02.412546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54329d14-7146-402e-a48d-8063bb7f5932 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:02.412814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54329d14-7146-402e-a48d-8063bb7f5932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:02.431986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43980064-b074-416b-831b-e17986a59fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:02.432824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43980064-b074-416b-831b-e17986a59fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:02.433251 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43980064-b074-416b-831b-e17986a59fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:02.433665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43980064-b074-416b-831b-e17986a59fd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:02.444726 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-54329d14-7146-402e-a48d-8063bb7f5932 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:02.444726 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-54329d14-7146-402e-a48d-8063bb7f5932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:02.542406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43980064-b074-416b-831b-e17986a59fd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:02.544438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43980064-b074-416b-831b-e17986a59fd6 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:02.549388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:02.555058 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 253/497] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:54:02.586490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:02.693773 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 245/498] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:54:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 18:54:02.699437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54329d14-7146-402e-a48d-8063bb7f5932 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:02.830632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54329d14-7146-402e-a48d-8063bb7f5932 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:02.866379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fc938-848c-44d5-abf7-0add3d5e92ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:02.867031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fc938-848c-44d5-abf7-0add3d5e92ed None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:02.867314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fc938-848c-44d5-abf7-0add3d5e92ed None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:02.867583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fc938-848c-44d5-abf7-0add3d5e92ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:02.877928 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204fc938-848c-44d5-abf7-0add3d5e92ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:02.879422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204fc938-848c-44d5-abf7-0add3d5e92ed None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:02.884265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43980064-b074-416b-831b-e17986a59fd6 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-204fc938-848c-44d5-abf7-0add3d5e92ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:02.894674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:02.895608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:02.896842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:02.896842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:02.896842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:02.896842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 254/499] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:02.904158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fc938-848c-44d5-abf7-0add3d5e92ed None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:02.959882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:03.012592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fc938-848c-44d5-abf7-0add3d5e92ed None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:03.069331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e663352-89b1-4dc5-ab42-2e406e1b37dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:03.071478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e663352-89b1-4dc5-ab42-2e406e1b37dd None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.071984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e663352-89b1-4dc5-ab42-2e406e1b37dd None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.073729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e663352-89b1-4dc5-ab42-2e406e1b37dd None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.074570 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e663352-89b1-4dc5-ab42-2e406e1b37dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:03.075446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e663352-89b1-4dc5-ab42-2e406e1b37dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:03.077230 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e663352-89b1-4dc5-ab42-2e406e1b37dd None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:03.110134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 255/500] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:03.113809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e663352-89b1-4dc5-ab42-2e406e1b37dd None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:03.118576 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43980064-b074-416b-831b-e17986a59fd6 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:03.165210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:03.187746 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d72dde-d335-40dc-a350-d22d229e26a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:03.188411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d72dde-d335-40dc-a350-d22d229e26a6 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.188688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d72dde-d335-40dc-a350-d22d229e26a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.188962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d72dde-d335-40dc-a350-d22d229e26a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.208360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d72dde-d335-40dc-a350-d22d229e26a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-e5d72dde-d335-40dc-a350-d22d229e26a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:03.213484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:03.214516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:03.216403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:03.218647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 246/501] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:54:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:03.223957 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d72dde-d335-40dc-a350-d22d229e26a6 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:03.326161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e663352-89b1-4dc5-ab42-2e406e1b37dd None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:03.338160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d72dde-d335-40dc-a350-d22d229e26a6 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:03.341530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:03.362579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213e550-e473-466e-bf2e-9a1c92db6cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:03.363330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e550-e473-466e-bf2e-9a1c92db6cec None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.363678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e550-e473-466e-bf2e-9a1c92db6cec None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.364018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e550-e473-466e-bf2e-9a1c92db6cec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.371118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213e550-e473-466e-bf2e-9a1c92db6cec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:03.372751 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213e550-e473-466e-bf2e-9a1c92db6cec None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:03.376936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854227-5136-4122-99a8-4f09af22717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:03.377621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854227-5136-4122-99a8-4f09af22717a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.377974 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854227-5136-4122-99a8-4f09af22717a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.378313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854227-5136-4122-99a8-4f09af22717a None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.379087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854227-5136-4122-99a8-4f09af22717a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:03.379470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854227-5136-4122-99a8-4f09af22717a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:03.380912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854227-5136-4122-99a8-4f09af22717a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-e213e550-e473-466e-bf2e-9a1c92db6cec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:03.385647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:03.386760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:03.388763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:03.388763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:03.388763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:03.388763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:03.388763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:03.388763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:03.388763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:03.388763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:03.389666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 256/502] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:03.390760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713d1df-bcb3-4af4-9eba-0010b3922a40 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.391625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713d1df-bcb3-4af4-9eba-0010b3922a40 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.391625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713d1df-bcb3-4af4-9eba-0010b3922a40 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.391830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 247/503] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:54:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:03.392824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 257/504] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:03.398247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713d1df-bcb3-4af4-9eba-0010b3922a40 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:03.398518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7764b31-8e7c-4708-8a1d-f2cac659bf1f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.398807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7764b31-8e7c-4708-8a1d-f2cac659bf1f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.399123 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7764b31-8e7c-4708-8a1d-f2cac659bf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.422771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f7764b31-8e7c-4708-8a1d-f2cac659bf1f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:03.424113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f7764b31-8e7c-4708-8a1d-f2cac659bf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:03.471119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:03.507190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 248/505] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 18:54:03.508376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7764b31-8e7c-4708-8a1d-f2cac659bf1f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:03.514166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713d1df-bcb3-4af4-9eba-0010b3922a40 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:03.555123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e5e6-7199-488d-940c-da3d113b5b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:03.555804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e5e6-7199-488d-940c-da3d113b5b58 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.558260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e5e6-7199-488d-940c-da3d113b5b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.558260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e5e6-7199-488d-940c-da3d113b5b58 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.558260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4e5e6-7199-488d-940c-da3d113b5b58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:03.558260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4e5e6-7199-488d-940c-da3d113b5b58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:03.558716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4e5e6-7199-488d-940c-da3d113b5b58 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:03.587025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 258/506] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:03.592270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e5e6-7199-488d-940c-da3d113b5b58 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:03.652559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:03.685784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7764b31-8e7c-4708-8a1d-f2cac659bf1f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:03.731458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26e605-49cf-4c37-aa6d-4d6c2df9eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:03.732308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26e605-49cf-4c37-aa6d-4d6c2df9eec8 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.732751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26e605-49cf-4c37-aa6d-4d6c2df9eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.733168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26e605-49cf-4c37-aa6d-4d6c2df9eec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.747211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26e605-49cf-4c37-aa6d-4d6c2df9eec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:03.748876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26e605-49cf-4c37-aa6d-4d6c2df9eec8 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:03.753717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e5e6-7199-488d-940c-da3d113b5b58 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-2d26e605-49cf-4c37-aa6d-4d6c2df9eec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:03.767180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:03.768306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:03.769659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:03.769659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:03.769659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:03.775384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 249/507] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:03.777104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9693c5c-48ea-43e4-9490-785e7c5768b5 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.777401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9693c5c-48ea-43e4-9490-785e7c5768b5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.777650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9693c5c-48ea-43e4-9490-785e7c5768b5 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.779019 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 250/508] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:03.782740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9693c5c-48ea-43e4-9490-785e7c5768b5 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:03.802965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5970ba-7939-4ac9-97c5-abe63b075549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:03.803867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5970ba-7939-4ac9-97c5-abe63b075549 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.804140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5970ba-7939-4ac9-97c5-abe63b075549 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.804402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5970ba-7939-4ac9-97c5-abe63b075549 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.805599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5970ba-7939-4ac9-97c5-abe63b075549 None admin] RBAC: Adding query filter params (role.id=01b71dd1f22e41269c3caa83dea57b2e, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=a51690d0367b45db8a6552764cc1d11e) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:03.805907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5970ba-7939-4ac9-97c5-abe63b075549 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:03.807620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5970ba-7939-4ac9-97c5-abe63b075549 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:03.821088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 259/509] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:03 2022] GET /identity/v3/role_assignments?user.id=a51690d0367b45db8a6552764cc1d11e&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=01b71dd1f22e41269c3caa83dea57b2e => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:03.822953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405827d-2a1e-499b-9e02-8ce97428ebd7 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.823500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405827d-2a1e-499b-9e02-8ce97428ebd7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.825583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405827d-2a1e-499b-9e02-8ce97428ebd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.832818 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:03.853100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e405827d-2a1e-499b-9e02-8ce97428ebd7 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:03.854678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e405827d-2a1e-499b-9e02-8ce97428ebd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:03.929608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9693c5c-48ea-43e4-9490-785e7c5768b5 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:03.930165 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 260/510] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:54:03.939349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405827d-2a1e-499b-9e02-8ce97428ebd7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:03.970531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb305f7-acf5-48cd-9ae1-b44b02c7b159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:03.971123 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb305f7-acf5-48cd-9ae1-b44b02c7b159 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:03.971461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb305f7-acf5-48cd-9ae1-b44b02c7b159 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:03.971735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb305f7-acf5-48cd-9ae1-b44b02c7b159 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:03.972394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb305f7-acf5-48cd-9ae1-b44b02c7b159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:03.972699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb305f7-acf5-48cd-9ae1-b44b02c7b159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:03.974162 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb305f7-acf5-48cd-9ae1-b44b02c7b159 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:03.990229 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:03.990417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 251/511] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:03.997085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb305f7-acf5-48cd-9ae1-b44b02c7b159 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:04.074192 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405827d-2a1e-499b-9e02-8ce97428ebd7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:04.091860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:04.103539 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb305f7-acf5-48cd-9ae1-b44b02c7b159 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:04.112111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df05bf-8079-4876-a0f0-5a2007cec7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:04.112855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df05bf-8079-4876-a0f0-5a2007cec7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.113379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df05bf-8079-4876-a0f0-5a2007cec7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.113743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df05bf-8079-4876-a0f0-5a2007cec7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.125131 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df05bf-8079-4876-a0f0-5a2007cec7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:04.126710 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df05bf-8079-4876-a0f0-5a2007cec7c7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:04.141257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadc443-9e85-4e22-aca9-5917aeefa359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:04.141881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadc443-9e85-4e22-aca9-5917aeefa359 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.142162 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadc443-9e85-4e22-aca9-5917aeefa359 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.142435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadc443-9e85-4e22-aca9-5917aeefa359 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-e4df05bf-8079-4876-a0f0-5a2007cec7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:04.144021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:04.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:04.146037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:04.146037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:04.146037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:04.146633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 261/512] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:04.154525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df05bf-8079-4876-a0f0-5a2007cec7c7 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:04.156254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadc443-9e85-4e22-aca9-5917aeefa359 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:04.157708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadc443-9e85-4e22-aca9-5917aeefa359 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-eeadc443-9e85-4e22-aca9-5917aeefa359 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:04.176362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:04.177336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:04.178262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 252/513] 10.222.0.58 () {56 vars in 1093 bytes} [Fri Oct 28 18:54:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:04.188274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadc443-9e85-4e22-aca9-5917aeefa359 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:04.263079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:04.287583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df05bf-8079-4876-a0f0-5a2007cec7c7 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:04.323774 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadc443-9e85-4e22-aca9-5917aeefa359 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:04.326444 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4233b0-989c-4634-bc40-639f1a35ced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:04.327252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4233b0-989c-4634-bc40-639f1a35ced5 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.327675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4233b0-989c-4634-bc40-639f1a35ced5 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.328091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4233b0-989c-4634-bc40-639f1a35ced5 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.328901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4233b0-989c-4634-bc40-639f1a35ced5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:04.329348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4233b0-989c-4634-bc40-639f1a35ced5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:04.330911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4233b0-989c-4634-bc40-639f1a35ced5 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:04.345075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 262/514] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:04.346268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4233b0-989c-4634-bc40-639f1a35ced5 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:04.360392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595336fe-70db-4513-b757-2ad7c4779658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:04.361130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595336fe-70db-4513-b757-2ad7c4779658 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.361503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595336fe-70db-4513-b757-2ad7c4779658 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.361888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595336fe-70db-4513-b757-2ad7c4779658 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.362704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595336fe-70db-4513-b757-2ad7c4779658 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:04.363148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595336fe-70db-4513-b757-2ad7c4779658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:04.364872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595336fe-70db-4513-b757-2ad7c4779658 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:04.383692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 253/515] 10.222.0.58 () {56 vars in 1133 bytes} [Fri Oct 28 18:54:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:04.384811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595336fe-70db-4513-b757-2ad7c4779658 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:04.393659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:04.462962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4233b0-989c-4634-bc40-639f1a35ced5 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:04.498549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595336fe-70db-4513-b757-2ad7c4779658 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:04.503076 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa96897-115f-437a-bb0e-01e1ed433c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:04.503891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa96897-115f-437a-bb0e-01e1ed433c55 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.504481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa96897-115f-437a-bb0e-01e1ed433c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.504923 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa96897-115f-437a-bb0e-01e1ed433c55 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.505653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa96897-115f-437a-bb0e-01e1ed433c55 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:04.507297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa96897-115f-437a-bb0e-01e1ed433c55 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-2aa96897-115f-437a-bb0e-01e1ed433c55 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:04.520367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:04.521482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 18:54:04.522396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:04.525619 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 263/516] 10.222.0.58 () {56 vars in 1129 bytes} [Fri Oct 28 18:54:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:04.526565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa96897-115f-437a-bb0e-01e1ed433c55 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:04.531300 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:04.539591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44088ab5-1c0c-4c26-a820-cc924a087cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:04.540244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44088ab5-1c0c-4c26-a820-cc924a087cfc None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.540622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44088ab5-1c0c-4c26-a820-cc924a087cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.541044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44088ab5-1c0c-4c26-a820-cc924a087cfc None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.542178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44088ab5-1c0c-4c26-a820-cc924a087cfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:04.544182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44088ab5-1c0c-4c26-a820-cc924a087cfc None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:04.565882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 254/517] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:54:04 2022] POST /identity/v3/users => generated 254 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:04.566548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44088ab5-1c0c-4c26-a820-cc924a087cfc None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:04.655902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:04.664946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa96897-115f-437a-bb0e-01e1ed433c55 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:04.706804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7179640-7407-4b79-87f0-c51586a9774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:04.707511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7179640-7407-4b79-87f0-c51586a9774b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.707874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7179640-7407-4b79-87f0-c51586a9774b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.708144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7179640-7407-4b79-87f0-c51586a9774b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.712199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44088ab5-1c0c-4c26-a820-cc924a087cfc None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:04.720820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7179640-7407-4b79-87f0-c51586a9774b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:04.722431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7179640-7407-4b79-87f0-c51586a9774b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-c7179640-7407-4b79-87f0-c51586a9774b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:04.744873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:04.745877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:04.746809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:04.746809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:04.746809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:04.746809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 264/518] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:04.749836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786c72a-d1c5-4a0f-ada0-34dc3ea2afec None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.749836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786c72a-d1c5-4a0f-ada0-34dc3ea2afec None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.749836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786c72a-d1c5-4a0f-ada0-34dc3ea2afec None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.750270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 265/519] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:04.754288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9f438-bb3a-4ccf-9703-7a303531a907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:04.754886 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9f438-bb3a-4ccf-9703-7a303531a907 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.755193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9f438-bb3a-4ccf-9703-7a303531a907 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.755468 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9f438-bb3a-4ccf-9703-7a303531a907 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.756098 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9f438-bb3a-4ccf-9703-7a303531a907 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:04.756367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc287d58-2ee7-4603-9f02-4bac1f216132 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.756481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9f438-bb3a-4ccf-9703-7a303531a907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:04.756803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc287d58-2ee7-4603-9f02-4bac1f216132 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.757172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc287d58-2ee7-4603-9f02-4bac1f216132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.757752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9f438-bb3a-4ccf-9703-7a303531a907 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:04.763419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 255/520] 10.222.0.58 () {56 vars in 1124 bytes} [Fri Oct 28 18:54:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:04.764418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9f438-bb3a-4ccf-9703-7a303531a907 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:04.784941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-fc287d58-2ee7-4603-9f02-4bac1f216132 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:04.786053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-fc287d58-2ee7-4603-9f02-4bac1f216132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:04.853536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:04.873608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 266/521] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:54:04.874462 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc287d58-2ee7-4603-9f02-4bac1f216132 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:04.897718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9f438-bb3a-4ccf-9703-7a303531a907 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:04.952404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:04.962636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af8634-917e-44a2-8256-abc296338201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:04.963395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af8634-917e-44a2-8256-abc296338201 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:04.963673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af8634-917e-44a2-8256-abc296338201 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:04.963946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af8634-917e-44a2-8256-abc296338201 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:04.964579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af8634-917e-44a2-8256-abc296338201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:04.964879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af8634-917e-44a2-8256-abc296338201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:04.967040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af8634-917e-44a2-8256-abc296338201 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:04.983822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 256/522] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:04.988121 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af8634-917e-44a2-8256-abc296338201 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:05.071093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc287d58-2ee7-4603-9f02-4bac1f216132 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:05.115561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0c543-856b-4107-abaf-fdd85c0e9f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:05.116170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0c543-856b-4107-abaf-fdd85c0e9f31 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:05.116304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af8634-917e-44a2-8256-abc296338201 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:05.116474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0c543-856b-4107-abaf-fdd85c0e9f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:05.116705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0c543-856b-4107-abaf-fdd85c0e9f31 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:05.117329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed0c543-856b-4107-abaf-fdd85c0e9f31 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:05.117884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed0c543-856b-4107-abaf-fdd85c0e9f31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:05.120160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed0c543-856b-4107-abaf-fdd85c0e9f31 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:05.127408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 267/523] 10.222.0.58 () {56 vars in 1124 bytes} [Fri Oct 28 18:54:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:05.188510 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85651ea6-bd29-470a-a718-ff636ff7282f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:05.189177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85651ea6-bd29-470a-a718-ff636ff7282f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:05.189419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85651ea6-bd29-470a-a718-ff636ff7282f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:05.189688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85651ea6-bd29-470a-a718-ff636ff7282f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:05.201345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85651ea6-bd29-470a-a718-ff636ff7282f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:05.202814 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85651ea6-bd29-470a-a718-ff636ff7282f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-85651ea6-bd29-470a-a718-ff636ff7282f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:05.214954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:05.216853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:05.218607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:05.218607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:05.218607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:05.218607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:05.218607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:05.218607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:05.218607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:05.218607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:05.218607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 257/524] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:05.225324 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0c543-856b-4107-abaf-fdd85c0e9f31 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:05.232694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85651ea6-bd29-470a-a718-ff636ff7282f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:05.317074 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:05.334563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:05.397643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85651ea6-bd29-470a-a718-ff636ff7282f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:05.422452 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0c543-856b-4107-abaf-fdd85c0e9f31 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:05.445862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952c7e8-f30d-4285-8097-301f8e3ebb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:05.446495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952c7e8-f30d-4285-8097-301f8e3ebb9a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:05.447065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952c7e8-f30d-4285-8097-301f8e3ebb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:05.447343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952c7e8-f30d-4285-8097-301f8e3ebb9a None admin] PATH_INFO: `/v3/services/network` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:05.448014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952c7e8-f30d-4285-8097-301f8e3ebb9a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:05.449699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952c7e8-f30d-4285-8097-301f8e3ebb9a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:05.462835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe363a2-f512-499e-9813-8901d6351494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-b952c7e8-f30d-4285-8097-301f8e3ebb9a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:05.463419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:05.464372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 18:54:05.465248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:05.466052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe363a2-f512-499e-9813-8901d6351494 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:05.466052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe363a2-f512-499e-9813-8901d6351494 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:05.466052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe363a2-f512-499e-9813-8901d6351494 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:05.466052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe363a2-f512-499e-9813-8901d6351494 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:05.466052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe363a2-f512-499e-9813-8901d6351494 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:05.466874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe363a2-f512-499e-9813-8901d6351494 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:05.472067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 258/525] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:05 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:05.472067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952c7e8-f30d-4285-8097-301f8e3ebb9a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:05.481786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 268/526] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:05.488005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe363a2-f512-499e-9813-8901d6351494 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:05.593120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952c7e8-f30d-4285-8097-301f8e3ebb9a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:05.624786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:05.671334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe363a2-f512-499e-9813-8901d6351494 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:05.681588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:05.699119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280caa3-bfc9-4663-8ae7-ca88ba6444d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:05.699779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280caa3-bfc9-4663-8ae7-ca88ba6444d4 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:05.700066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280caa3-bfc9-4663-8ae7-ca88ba6444d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:05.700338 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280caa3-bfc9-4663-8ae7-ca88ba6444d4 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:05.700976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280caa3-bfc9-4663-8ae7-ca88ba6444d4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:05.701280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280caa3-bfc9-4663-8ae7-ca88ba6444d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:05.702668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280caa3-bfc9-4663-8ae7-ca88ba6444d4 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:05.711040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 259/527] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:05.716641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280caa3-bfc9-4663-8ae7-ca88ba6444d4 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:05.717680 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a94ddb-3ef7-475f-8f8c-7bbd6521db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:05.718339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a94ddb-3ef7-475f-8f8c-7bbd6521db5f None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:05.718614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a94ddb-3ef7-475f-8f8c-7bbd6521db5f None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:05.718887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a94ddb-3ef7-475f-8f8c-7bbd6521db5f None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a758611cf8d94e8ca8aad6a75bef5e34/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:05.723262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a94ddb-3ef7-475f-8f8c-7bbd6521db5f None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=a758611cf8d94e8ca8aad6a75bef5e34, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:05.724689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a94ddb-3ef7-475f-8f8c-7bbd6521db5f None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:05.764405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 269/528] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:54:05 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a758611cf8d94e8ca8aad6a75bef5e34/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:54:05.880893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280caa3-bfc9-4663-8ae7-ca88ba6444d4 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:05.933676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debce50-866b-45f9-b89f-9eaa82484faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:05.934355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debce50-866b-45f9-b89f-9eaa82484faf None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:05.934641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debce50-866b-45f9-b89f-9eaa82484faf None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:05.934916 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debce50-866b-45f9-b89f-9eaa82484faf None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:05.935596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debce50-866b-45f9-b89f-9eaa82484faf None admin] RBAC: Adding query filter params (type=network) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:05.935901 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debce50-866b-45f9-b89f-9eaa82484faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:05.937424 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debce50-866b-45f9-b89f-9eaa82484faf None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:05.948143 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 260/529] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:06.608943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc94b9-efb7-4ac0-b638-a4ff2fdf1726 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:06.609322 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc94b9-efb7-4ac0-b638-a4ff2fdf1726 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:06.609500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc94b9-efb7-4ac0-b638-a4ff2fdf1726 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:06.611111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 270/530] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:06.616595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b23d4f-6a28-424c-847c-34e3a9f66a75 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:06.617058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b23d4f-6a28-424c-847c-34e3a9f66a75 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:06.617152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b23d4f-6a28-424c-847c-34e3a9f66a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:06.644218 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-60b23d4f-6a28-424c-847c-34e3a9f66a75 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:06.645415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-60b23d4f-6a28-424c-847c-34e3a9f66a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:06.684697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:06.741634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 261/531] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:54:07.058547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc94b9-efb7-4ac0-b638-a4ff2fdf1726 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:07.120981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6ef53-0237-432c-b6c1-5ad02d101cc5 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:07.121306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6ef53-0237-432c-b6c1-5ad02d101cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:07.122132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6ef53-0237-432c-b6c1-5ad02d101cc5 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:07.124464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 262/532] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:07.128232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aafcd-e55d-4b6b-b248-3dc903bef03c None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:07.128442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aafcd-e55d-4b6b-b248-3dc903bef03c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:07.128717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aafcd-e55d-4b6b-b248-3dc903bef03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:07.164051 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-862aafcd-e55d-4b6b-b248-3dc903bef03c None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:07.165693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-862aafcd-e55d-4b6b-b248-3dc903bef03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:07.180671 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc94b9-efb7-4ac0-b638-a4ff2fdf1726 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:07.216565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec79c1-10dd-4c6a-81e7-12615cec093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:07.217516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec79c1-10dd-4c6a-81e7-12615cec093e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:07.218021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec79c1-10dd-4c6a-81e7-12615cec093e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:07.218494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec79c1-10dd-4c6a-81e7-12615cec093e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:07.229087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec79c1-10dd-4c6a-81e7-12615cec093e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-f7ec79c1-10dd-4c6a-81e7-12615cec093e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:07.234920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:07.235893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:07.236716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:07.239298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 271/533] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:54:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:07.245251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec79c1-10dd-4c6a-81e7-12615cec093e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:07.290079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:07.316666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 263/534] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:54:07.364015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec79c1-10dd-4c6a-81e7-12615cec093e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:07.399546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af838e-3b1c-4358-ac6d-766cb414ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:07.400507 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af838e-3b1c-4358-ac6d-766cb414ac9d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:07.400978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af838e-3b1c-4358-ac6d-766cb414ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:07.401477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af838e-3b1c-4358-ac6d-766cb414ac9d None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:07.402321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af838e-3b1c-4358-ac6d-766cb414ac9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:07.403362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af838e-3b1c-4358-ac6d-766cb414ac9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:07.405056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af838e-3b1c-4358-ac6d-766cb414ac9d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:07.413767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 272/535] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:54:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:07.424085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862aafcd-e55d-4b6b-b248-3dc903bef03c None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:07.581386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:07.584929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862aafcd-e55d-4b6b-b248-3dc903bef03c None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:07.628734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4760d9-21c0-47d2-b220-f24ad6ac87fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:07.629385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4760d9-21c0-47d2-b220-f24ad6ac87fa None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:07.629629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4760d9-21c0-47d2-b220-f24ad6ac87fa None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:07.629899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4760d9-21c0-47d2-b220-f24ad6ac87fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:07.650450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4760d9-21c0-47d2-b220-f24ad6ac87fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:07.651924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4760d9-21c0-47d2-b220-f24ad6ac87fa None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-4f4760d9-21c0-47d2-b220-f24ad6ac87fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:07.670862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:07.671867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:07.672744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:07.672744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:07.672744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:07.672744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 264/536] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:07.677943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af838e-3b1c-4358-ac6d-766cb414ac9d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:07.837843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af838e-3b1c-4358-ac6d-766cb414ac9d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:07.847684 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:07.890402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38dda0-9906-4e1f-9472-e77c5c0eda53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:07.891399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38dda0-9906-4e1f-9472-e77c5c0eda53 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:07.891899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38dda0-9906-4e1f-9472-e77c5c0eda53 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:07.892360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38dda0-9906-4e1f-9472-e77c5c0eda53 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:07.893191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38dda0-9906-4e1f-9472-e77c5c0eda53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:07.893948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38dda0-9906-4e1f-9472-e77c5c0eda53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:07.895531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38dda0-9906-4e1f-9472-e77c5c0eda53 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:07.912496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 273/537] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:07.917671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4760d9-21c0-47d2-b220-f24ad6ac87fa None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:07.988145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38dda0-9906-4e1f-9472-e77c5c0eda53 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:08.066180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4760d9-21c0-47d2-b220-f24ad6ac87fa None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:08.101940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773d214-1bdc-4b42-bbe0-5cc4302dbdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:08.103016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d214-1bdc-4b42-bbe0-5cc4302dbdbf None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.103494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d214-1bdc-4b42-bbe0-5cc4302dbdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.103972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d214-1bdc-4b42-bbe0-5cc4302dbdbf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.104235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38dda0-9906-4e1f-9472-e77c5c0eda53 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:08.112741 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773d214-1bdc-4b42-bbe0-5cc4302dbdbf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:08.114558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773d214-1bdc-4b42-bbe0-5cc4302dbdbf None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-b773d214-1bdc-4b42-bbe0-5cc4302dbdbf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:08.127106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:08.128028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 18:54:08.128941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:08.131497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 265/538] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:54:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:08.132963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c1ef-dce3-489a-813d-a3f3e52bb9db None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.133451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c1ef-dce3-489a-813d-a3f3e52bb9db None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.133910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c1ef-dce3-489a-813d-a3f3e52bb9db None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.135596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 266/539] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:08.136793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3c1ef-dce3-489a-813d-a3f3e52bb9db None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:08.149281 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88bc6a-53be-45ea-9f46-199fe06869c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:08.149918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88bc6a-53be-45ea-9f46-199fe06869c8 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.150198 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88bc6a-53be-45ea-9f46-199fe06869c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.150526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88bc6a-53be-45ea-9f46-199fe06869c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.154902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:08.161514 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88bc6a-53be-45ea-9f46-199fe06869c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-6b88bc6a-53be-45ea-9f46-199fe06869c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:08.166789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:08.167718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:08.168588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 274/540] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:54:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:08.174400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a971e-9fbf-4cf0-99d5-17c03626f8ce None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.174400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a971e-9fbf-4cf0-99d5-17c03626f8ce None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.174400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a971e-9fbf-4cf0-99d5-17c03626f8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.190574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-912a971e-9fbf-4cf0-99d5-17c03626f8ce None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:08.191797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-912a971e-9fbf-4cf0-99d5-17c03626f8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:08.194731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:08.259008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3c1ef-dce3-489a-813d-a3f3e52bb9db None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:08.263060 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 275/541] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:54:08.268155 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a971e-9fbf-4cf0-99d5-17c03626f8ce None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:08.325579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736a074-a564-4dbd-ade6-2186be247273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:08.326721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736a074-a564-4dbd-ade6-2186be247273 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.328058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736a074-a564-4dbd-ade6-2186be247273 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.329054 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736a074-a564-4dbd-ade6-2186be247273 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.330141 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c736a074-a564-4dbd-ade6-2186be247273 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:08.332425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c736a074-a564-4dbd-ade6-2186be247273 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:08.334114 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c736a074-a564-4dbd-ade6-2186be247273 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:08.369211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 267/542] 10.222.0.58 () {56 vars in 1135 bytes} [Fri Oct 28 18:54:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:08.377634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736a074-a564-4dbd-ade6-2186be247273 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:08.467021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a971e-9fbf-4cf0-99d5-17c03626f8ce None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:08.492202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:08.504857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178c0dd-f349-40f7-9365-84ed7d9463da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:08.505474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178c0dd-f349-40f7-9365-84ed7d9463da None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.505756 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178c0dd-f349-40f7-9365-84ed7d9463da None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.506030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178c0dd-f349-40f7-9365-84ed7d9463da None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.506697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178c0dd-f349-40f7-9365-84ed7d9463da None admin] RBAC: Adding query filter params (name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:08.507010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178c0dd-f349-40f7-9365-84ed7d9463da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:08.508392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178c0dd-f349-40f7-9365-84ed7d9463da None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:08.522862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 276/543] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:54:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:08.524430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78b674-bc8f-4536-b99d-16eae751c211 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.524925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78b674-bc8f-4536-b99d-16eae751c211 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.525406 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78b674-bc8f-4536-b99d-16eae751c211 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.530891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 277/544] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:08.530891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78b674-bc8f-4536-b99d-16eae751c211 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:08.533883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:08.557555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736a074-a564-4dbd-ade6-2186be247273 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:08.593208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a02ff-61af-4624-93fc-c14f33130336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:08.593830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a02ff-61af-4624-93fc-c14f33130336 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.594111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a02ff-61af-4624-93fc-c14f33130336 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.594377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a02ff-61af-4624-93fc-c14f33130336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.603994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a02ff-61af-4624-93fc-c14f33130336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:08.605436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a02ff-61af-4624-93fc-c14f33130336 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-4a1a02ff-61af-4624-93fc-c14f33130336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:08.620185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:08.621334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:08.622766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:08.622766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:08.622766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:08.622766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 268/545] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:08.623399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4583b74-fcb2-41b3-94dd-e173fcd686b4 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.623587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4583b74-fcb2-41b3-94dd-e173fcd686b4 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.623858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4583b74-fcb2-41b3-94dd-e173fcd686b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.638295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78b674-bc8f-4536-b99d-16eae751c211 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:08.641788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d4583b74-fcb2-41b3-94dd-e173fcd686b4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:08.642951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d4583b74-fcb2-41b3-94dd-e173fcd686b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:08.671365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9ee09-9cc4-4e7f-b8d7-6b7b85082480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:08.671989 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ee09-9cc4-4e7f-b8d7-6b7b85082480 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.672269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ee09-9cc4-4e7f-b8d7-6b7b85082480 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.672544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ee09-9cc4-4e7f-b8d7-6b7b85082480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.682446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9ee09-9cc4-4e7f-b8d7-6b7b85082480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:08.684543 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9ee09-9cc4-4e7f-b8d7-6b7b85082480 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-dac9ee09-9cc4-4e7f-b8d7-6b7b85082480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:08.698809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:08.699907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:08.700763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:08.700763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:08.700763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:08.700763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 278/546] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:08.701184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2d183-1b56-4c2b-a788-908e0688945a None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.701369 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2d183-1b56-4c2b-a788-908e0688945a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.701638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2d183-1b56-4c2b-a788-908e0688945a None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.702872 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 279/547] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:08.703571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2d183-1b56-4c2b-a788-908e0688945a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:08.716436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 269/548] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:54:08.717275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4583b74-fcb2-41b3-94dd-e173fcd686b4 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:08.766141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:08.779505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:08.817525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2d183-1b56-4c2b-a788-908e0688945a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:08.830200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4583b74-fcb2-41b3-94dd-e173fcd686b4 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:08.854239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dcca5-8850-466c-ab19-5be62cdcf7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:08.855068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dcca5-8850-466c-ab19-5be62cdcf7b9 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.855196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dcca5-8850-466c-ab19-5be62cdcf7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.855394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dcca5-8850-466c-ab19-5be62cdcf7b9 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.856588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47dcca5-8850-466c-ab19-5be62cdcf7b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:08.857957 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47dcca5-8850-466c-ab19-5be62cdcf7b9 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:08.870375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2fd28-9c04-4380-9796-9abf9093e57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:08.870947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2fd28-9c04-4380-9796-9abf9093e57a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.871319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2fd28-9c04-4380-9796-9abf9093e57a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.871546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2fd28-9c04-4380-9796-9abf9093e57a None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.872187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2fd28-9c04-4380-9796-9abf9093e57a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:08.872492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2fd28-9c04-4380-9796-9abf9093e57a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:08.873846 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2fd28-9c04-4380-9796-9abf9093e57a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:08.879087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 280/549] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:54:08 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:08.879087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dcca5-8850-466c-ab19-5be62cdcf7b9 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:08.892632 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 270/550] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:08.894640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaae56-dcb2-46c1-a676-162b90538d91 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:08.894927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaae56-dcb2-46c1-a676-162b90538d91 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:08.895244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaae56-dcb2-46c1-a676-162b90538d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:08.924185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-70eaae56-dcb2-46c1-a676-162b90538d91 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:08.926404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-70eaae56-dcb2-46c1-a676-162b90538d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:09.019895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 271/551] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 18:54:09.020405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dcca5-8850-466c-ab19-5be62cdcf7b9 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.021351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaae56-dcb2-46c1-a676-162b90538d91 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.039681 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:09.069751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:09.070867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae13f8-2301-404a-b776-b33a01082a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.071376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae13f8-2301-404a-b776-b33a01082a1a None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.071647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae13f8-2301-404a-b776-b33a01082a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.071940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae13f8-2301-404a-b776-b33a01082a1a None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.072567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae13f8-2301-404a-b776-b33a01082a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:09.072846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae13f8-2301-404a-b776-b33a01082a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:09.074215 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae13f8-2301-404a-b776-b33a01082a1a None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:09.094596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 281/552] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:09.094596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae13f8-2301-404a-b776-b33a01082a1a None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.162924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaae56-dcb2-46c1-a676-162b90538d91 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.212374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5befae-68dd-4b57-b4a5-f15d0e5b60a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.212613 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae13f8-2301-404a-b776-b33a01082a1a None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.213044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5befae-68dd-4b57-b4a5-f15d0e5b60a4 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.213284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5befae-68dd-4b57-b4a5-f15d0e5b60a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.213540 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5befae-68dd-4b57-b4a5-f15d0e5b60a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.223220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5befae-68dd-4b57-b4a5-f15d0e5b60a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:09.225104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5befae-68dd-4b57-b4a5-f15d0e5b60a4 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-de5befae-68dd-4b57-b4a5-f15d0e5b60a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:09.238876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:09.239842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:09.240730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:09.240730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:09.240730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:09.240730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:09.240730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:09.240730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:09.240730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:09.240730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:09.240730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 272/553] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:09.241213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5befae-68dd-4b57-b4a5-f15d0e5b60a4 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.254805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568b571-e146-45e2-bff9-32db2c96f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.255516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568b571-e146-45e2-bff9-32db2c96f601 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.255787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568b571-e146-45e2-bff9-32db2c96f601 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.256060 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568b571-e146-45e2-bff9-32db2c96f601 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.270905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568b571-e146-45e2-bff9-32db2c96f601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-b568b571-e146-45e2-bff9-32db2c96f601 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:09.278706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:09.281138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 282/554] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:54:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:09.283184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568b571-e146-45e2-bff9-32db2c96f601 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.320960 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:09.347609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:09.354329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5befae-68dd-4b57-b4a5-f15d0e5b60a4 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.397058 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568b571-e146-45e2-bff9-32db2c96f601 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.400230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c17c6-ce99-4750-baf4-0e52dbb9cd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.400864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c17c6-ce99-4750-baf4-0e52dbb9cd90 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.401145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c17c6-ce99-4750-baf4-0e52dbb9cd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.401416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c17c6-ce99-4750-baf4-0e52dbb9cd90 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.409736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c17c6-ce99-4750-baf4-0e52dbb9cd90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:09.411200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c17c6-ce99-4750-baf4-0e52dbb9cd90 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-ef7c17c6-ce99-4750-baf4-0e52dbb9cd90 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:09.422417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:09.423450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 273/555] 10.222.0.58 () {56 vars in 1117 bytes} [Fri Oct 28 18:54:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:09.424878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c17c6-ce99-4750-baf4-0e52dbb9cd90 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.437041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64003f0-af49-491c-8233-9e6ff80c2879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.437640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64003f0-af49-491c-8233-9e6ff80c2879 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.437913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64003f0-af49-491c-8233-9e6ff80c2879 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.438181 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64003f0-af49-491c-8233-9e6ff80c2879 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.438865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64003f0-af49-491c-8233-9e6ff80c2879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:09.439190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64003f0-af49-491c-8233-9e6ff80c2879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:09.440627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64003f0-af49-491c-8233-9e6ff80c2879 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:09.452592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 283/556] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:09.453377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64003f0-af49-491c-8233-9e6ff80c2879 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.542793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c17c6-ce99-4750-baf4-0e52dbb9cd90 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.548860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:09.559624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64003f0-af49-491c-8233-9e6ff80c2879 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.580557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc8428-fa9d-496d-a889-0079f2ad571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.581230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc8428-fa9d-496d-a889-0079f2ad571d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.581574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc8428-fa9d-496d-a889-0079f2ad571d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.581849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc8428-fa9d-496d-a889-0079f2ad571d None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.582497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc8428-fa9d-496d-a889-0079f2ad571d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:09.582795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc8428-fa9d-496d-a889-0079f2ad571d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:09.584186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc8428-fa9d-496d-a889-0079f2ad571d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:09.592508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 274/557] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:54:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:09.593105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc8428-fa9d-496d-a889-0079f2ad571d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.599568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48a066-8c3e-43b6-95cc-44648088a99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.600184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48a066-8c3e-43b6-95cc-44648088a99a None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.600465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48a066-8c3e-43b6-95cc-44648088a99a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.600736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48a066-8c3e-43b6-95cc-44648088a99a None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.601385 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48a066-8c3e-43b6-95cc-44648088a99a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:09.602773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48a066-8c3e-43b6-95cc-44648088a99a None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:09.612647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:09.623027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 284/558] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:54:09 2022] POST /identity/v3/services => generated 240 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:09.623027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48a066-8c3e-43b6-95cc-44648088a99a None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.719160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc8428-fa9d-496d-a889-0079f2ad571d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.767193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48a066-8c3e-43b6-95cc-44648088a99a None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.770081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09d15f-4cd2-415e-86e0-b0421af5738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.770947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09d15f-4cd2-415e-86e0-b0421af5738c None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.771476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09d15f-4cd2-415e-86e0-b0421af5738c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.771920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09d15f-4cd2-415e-86e0-b0421af5738c None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.772851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09d15f-4cd2-415e-86e0-b0421af5738c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:09.773336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09d15f-4cd2-415e-86e0-b0421af5738c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:09.775033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09d15f-4cd2-415e-86e0-b0421af5738c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:09.795865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 275/559] 10.222.0.58 () {56 vars in 1149 bytes} [Fri Oct 28 18:54:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:09.797082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09d15f-4cd2-415e-86e0-b0421af5738c None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.820018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be6cc2-240a-45ba-8c3d-aac6cb254b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.820683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be6cc2-240a-45ba-8c3d-aac6cb254b50 None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.820961 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be6cc2-240a-45ba-8c3d-aac6cb254b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.821232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be6cc2-240a-45ba-8c3d-aac6cb254b50 None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a51690d0367b45db8a6552764cc1d11e/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.825251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be6cc2-240a-45ba-8c3d-aac6cb254b50 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=a51690d0367b45db8a6552764cc1d11e, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:09.826893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be6cc2-240a-45ba-8c3d-aac6cb254b50 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:09.845080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 285/560] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:54:09 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a51690d0367b45db8a6552764cc1d11e/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:54:09.846121 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be6cc2-240a-45ba-8c3d-aac6cb254b50 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:09.856895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:09.913331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:09.939594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09d15f-4cd2-415e-86e0-b0421af5738c None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:09.986751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d8ffb-3991-41f6-96e5-5e0a25cfdd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:09.987770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d8ffb-3991-41f6-96e5-5e0a25cfdd0a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:09.988149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d8ffb-3991-41f6-96e5-5e0a25cfdd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:09.988519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d8ffb-3991-41f6-96e5-5e0a25cfdd0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:09.993237 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be6cc2-240a-45ba-8c3d-aac6cb254b50 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:10.003699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d8ffb-3991-41f6-96e5-5e0a25cfdd0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:10.005718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d8ffb-3991-41f6-96e5-5e0a25cfdd0a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-9e8d8ffb-3991-41f6-96e5-5e0a25cfdd0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:10.028762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:10.029767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:10.030637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:10.030637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:10.030637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:10.030637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 276/561] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:10.036514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d8ffb-3991-41f6-96e5-5e0a25cfdd0a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:10.037196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c34ec2-6b00-42d4-8e18-14819dcfb2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:10.038054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c34ec2-6b00-42d4-8e18-14819dcfb2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:10.038538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c34ec2-6b00-42d4-8e18-14819dcfb2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:10.039026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c34ec2-6b00-42d4-8e18-14819dcfb2e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:10.060629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c34ec2-6b00-42d4-8e18-14819dcfb2e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:10.062488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c34ec2-6b00-42d4-8e18-14819dcfb2e8 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-d7c34ec2-6b00-42d4-8e18-14819dcfb2e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:10.085457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:10.086437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:10.087337 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:10.087337 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:10.087337 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:10.087337 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 286/562] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:10.092575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c34ec2-6b00-42d4-8e18-14819dcfb2e8 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:10.176509 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:10.199708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d8ffb-3991-41f6-96e5-5e0a25cfdd0a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:10.234019 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c34ec2-6b00-42d4-8e18-14819dcfb2e8 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:10.235849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:10.249038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f47698-a508-4494-b8ab-8753bab44b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:10.249848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f47698-a508-4494-b8ab-8753bab44b2a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:10.250222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f47698-a508-4494-b8ab-8753bab44b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:10.250609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f47698-a508-4494-b8ab-8753bab44b2a None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:10.251548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f47698-a508-4494-b8ab-8753bab44b2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:10.251950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f47698-a508-4494-b8ab-8753bab44b2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:10.254160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f47698-a508-4494-b8ab-8753bab44b2a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:10.269856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 277/563] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:10.274709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f47698-a508-4494-b8ab-8753bab44b2a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:10.288680 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5d3f6-fe68-4b94-aa29-7d8e1e5bc4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:10.289633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5d3f6-fe68-4b94-aa29-7d8e1e5bc4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:10.290187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5d3f6-fe68-4b94-aa29-7d8e1e5bc4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:10.290653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5d3f6-fe68-4b94-aa29-7d8e1e5bc4c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:10.291514 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5d3f6-fe68-4b94-aa29-7d8e1e5bc4c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:10.292020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5d3f6-fe68-4b94-aa29-7d8e1e5bc4c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:10.296079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5d3f6-fe68-4b94-aa29-7d8e1e5bc4c7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:10.322053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 287/564] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:10.327021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5d3f6-fe68-4b94-aa29-7d8e1e5bc4c7 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:10.490881 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5d3f6-fe68-4b94-aa29-7d8e1e5bc4c7 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:10.509893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f47698-a508-4494-b8ab-8753bab44b2a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:10.510848 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:10.532888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aef82d-666e-4d0c-8a35-c7f8ed1db8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:10.533522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aef82d-666e-4d0c-8a35-c7f8ed1db8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:10.533798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aef82d-666e-4d0c-8a35-c7f8ed1db8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:10.534062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aef82d-666e-4d0c-8a35-c7f8ed1db8a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:10.550405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aef82d-666e-4d0c-8a35-c7f8ed1db8a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:10.551887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aef82d-666e-4d0c-8a35-c7f8ed1db8a9 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-b3aef82d-666e-4d0c-8a35-c7f8ed1db8a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:10.569527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:10.570474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:10.571398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:10.571398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:10.571398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:10.571398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:10.571398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:10.571398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:10.571398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:10.571398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:10.577473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 288/565] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:10.583915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aef82d-666e-4d0c-8a35-c7f8ed1db8a9 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:10.585725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70fe65-a394-4c38-8cad-e42ba934ff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:10.586924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70fe65-a394-4c38-8cad-e42ba934ff7c None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:10.588412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70fe65-a394-4c38-8cad-e42ba934ff7c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:10.588901 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70fe65-a394-4c38-8cad-e42ba934ff7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:10.602526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70fe65-a394-4c38-8cad-e42ba934ff7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:10.609628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70fe65-a394-4c38-8cad-e42ba934ff7c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-8b70fe65-a394-4c38-8cad-e42ba934ff7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:10.627014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:10.628003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:54:10.628952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:10.629574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 278/566] 10.222.0.58 () {56 vars in 1093 bytes} [Fri Oct 28 18:54:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:10.638454 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70fe65-a394-4c38-8cad-e42ba934ff7c None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:10.733823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:10.758236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aef82d-666e-4d0c-8a35-c7f8ed1db8a9 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:10.816561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfd179-4660-4e99-8fe5-eecc801fd5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:10.817181 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfd179-4660-4e99-8fe5-eecc801fd5ea None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:10.817560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfd179-4660-4e99-8fe5-eecc801fd5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:10.817835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfd179-4660-4e99-8fe5-eecc801fd5ea None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:10.818530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfd179-4660-4e99-8fe5-eecc801fd5ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:10.818836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfd179-4660-4e99-8fe5-eecc801fd5ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:10.820297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfd179-4660-4e99-8fe5-eecc801fd5ea None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:10.833778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 289/567] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:10.840320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfd179-4660-4e99-8fe5-eecc801fd5ea None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:10.857718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70fe65-a394-4c38-8cad-e42ba934ff7c None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:10.885275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:10.928577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7998a26-d410-4bb1-9721-b2b7250d0cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:10.929191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7998a26-d410-4bb1-9721-b2b7250d0cbd None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:10.929472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7998a26-d410-4bb1-9721-b2b7250d0cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:10.929747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7998a26-d410-4bb1-9721-b2b7250d0cbd None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:10.930476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7998a26-d410-4bb1-9721-b2b7250d0cbd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:10.930779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7998a26-d410-4bb1-9721-b2b7250d0cbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:10.932241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7998a26-d410-4bb1-9721-b2b7250d0cbd None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:10.952876 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfd179-4660-4e99-8fe5-eecc801fd5ea None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:10.969409 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 279/568] 10.222.0.58 () {56 vars in 1133 bytes} [Fri Oct 28 18:54:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:10.976295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7998a26-d410-4bb1-9721-b2b7250d0cbd None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:11.000675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae02bc8-cb7c-448b-9484-3e14f64f19b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:11.002236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae02bc8-cb7c-448b-9484-3e14f64f19b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:11.002516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae02bc8-cb7c-448b-9484-3e14f64f19b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:11.002790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae02bc8-cb7c-448b-9484-3e14f64f19b6 None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/0eef740ccf884c079e6a2d6cdc89f4bb/roles/01b71dd1f22e41269c3caa83dea57b2e` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:11.034524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae02bc8-cb7c-448b-9484-3e14f64f19b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=0eef740ccf884c079e6a2d6cdc89f4bb, role_id=01b71dd1f22e41269c3caa83dea57b2e)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:11.035995 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae02bc8-cb7c-448b-9484-3e14f64f19b6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:11.062730 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 290/569] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:54:10 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/0eef740ccf884c079e6a2d6cdc89f4bb/roles/01b71dd1f22e41269c3caa83dea57b2e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 18:54:11.148830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7998a26-d410-4bb1-9721-b2b7250d0cbd None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:11.158274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:11.191600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a836c-bdd3-4631-b20e-8ee93bd84379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:11.192325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a836c-bdd3-4631-b20e-8ee93bd84379 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:11.192612 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a836c-bdd3-4631-b20e-8ee93bd84379 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:11.192889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a836c-bdd3-4631-b20e-8ee93bd84379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:11.271238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a836c-bdd3-4631-b20e-8ee93bd84379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:11.272784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a836c-bdd3-4631-b20e-8ee93bd84379 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:11.316365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed4a46-6c0c-42f8-ba23-c98d1ae2968f None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:11.316887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed4a46-6c0c-42f8-ba23-c98d1ae2968f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:11.316887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed4a46-6c0c-42f8-ba23-c98d1ae2968f None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:11.318326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 291/570] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:11.325810 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659ca95-f491-4c97-b143-fd22e1e1db59 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:11.326106 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659ca95-f491-4c97-b143-fd22e1e1db59 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:11.326365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659ca95-f491-4c97-b143-fd22e1e1db59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:11.354714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a659ca95-f491-4c97-b143-fd22e1e1db59 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:11.356521 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a659ca95-f491-4c97-b143-fd22e1e1db59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:11.417437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 280/571] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:54:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:54:11.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a836c-bdd3-4631-b20e-8ee93bd84379 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:11.439325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:11.461243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 292/572] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:11.578673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a836c-bdd3-4631-b20e-8ee93bd84379 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:11.615222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e34c82-294b-4692-b17e-e986560a3669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:11.615909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e34c82-294b-4692-b17e-e986560a3669 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:11.616192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e34c82-294b-4692-b17e-e986560a3669 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:11.616461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e34c82-294b-4692-b17e-e986560a3669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:11.627285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e34c82-294b-4692-b17e-e986560a3669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:11.628753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e34c82-294b-4692-b17e-e986560a3669 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-79e34c82-294b-4692-b17e-e986560a3669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:11.645221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:11.646231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:11.647131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:11.647131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:11.647131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:11.647131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 281/573] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:11.652159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659ca95-f491-4c97-b143-fd22e1e1db59 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:11.672244 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:11.771715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659ca95-f491-4c97-b143-fd22e1e1db59 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:11.806229 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff82eec-e046-4f61-b068-468072c78f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:11.806937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff82eec-e046-4f61-b068-468072c78f0c None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:11.807311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff82eec-e046-4f61-b068-468072c78f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:11.807590 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff82eec-e046-4f61-b068-468072c78f0c None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:11.808231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff82eec-e046-4f61-b068-468072c78f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:11.808534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff82eec-e046-4f61-b068-468072c78f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:11.810449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff82eec-e046-4f61-b068-468072c78f0c None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:11.826705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 293/574] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:11.832164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e34c82-294b-4692-b17e-e986560a3669 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:11.884160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff82eec-e046-4f61-b068-468072c78f0c None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:11.913766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:11.994884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e34c82-294b-4692-b17e-e986560a3669 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:11.998066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:12.037554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff82eec-e046-4f61-b068-468072c78f0c None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:12.060982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccae21-69d5-49bf-b610-334cf93df336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:12.061626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccae21-69d5-49bf-b610-334cf93df336 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.061908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccae21-69d5-49bf-b610-334cf93df336 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.062182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccae21-69d5-49bf-b610-334cf93df336 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.079436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccae21-69d5-49bf-b610-334cf93df336 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:12.080963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccae21-69d5-49bf-b610-334cf93df336 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-83ccae21-69d5-49bf-b610-334cf93df336 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:12.099254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:12.100191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:12.101069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:12.101069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:12.101069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:12.101069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:12.101069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:12.101069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:12.101069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:12.101069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:12.101069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 282/575] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:12.107220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccae21-69d5-49bf-b610-334cf93df336 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:12.116576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b939d52-7a20-4f42-a642-f4d1882d6511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:12.117212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b939d52-7a20-4f42-a642-f4d1882d6511 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.117488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b939d52-7a20-4f42-a642-f4d1882d6511 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.117753 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b939d52-7a20-4f42-a642-f4d1882d6511 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.118337 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b939d52-7a20-4f42-a642-f4d1882d6511 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:12.119703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b939d52-7a20-4f42-a642-f4d1882d6511 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-1b939d52-7a20-4f42-a642-f4d1882d6511 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:12.140981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:12.142068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 18:54:12.142921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:12.151016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 294/576] 10.222.0.58 () {56 vars in 1129 bytes} [Fri Oct 28 18:54:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:12.152382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b939d52-7a20-4f42-a642-f4d1882d6511 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:12.222546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccae21-69d5-49bf-b610-334cf93df336 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:12.242322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:12.264003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5e5fe-e694-47fd-b75b-4b82cc167f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:12.265067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5e5fe-e694-47fd-b75b-4b82cc167f89 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.265605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5e5fe-e694-47fd-b75b-4b82cc167f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.266198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5e5fe-e694-47fd-b75b-4b82cc167f89 None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.267194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5e5fe-e694-47fd-b75b-4b82cc167f89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:12.267752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5e5fe-e694-47fd-b75b-4b82cc167f89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:12.269828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5e5fe-e694-47fd-b75b-4b82cc167f89 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:12.284320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 283/577] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:12.285805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea97a-27a3-4e83-93d3-cf87ed73d1a9 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.286091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea97a-27a3-4e83-93d3-cf87ed73d1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.286362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea97a-27a3-4e83-93d3-cf87ed73d1a9 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.287801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 284/578] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:12.291125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ea97a-27a3-4e83-93d3-cf87ed73d1a9 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:12.336028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b939d52-7a20-4f42-a642-f4d1882d6511 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:12.376792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fecf5-909f-4030-858e-2ddd33149720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:12.377414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fecf5-909f-4030-858e-2ddd33149720 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.377697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fecf5-909f-4030-858e-2ddd33149720 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.377966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fecf5-909f-4030-858e-2ddd33149720 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.378596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fecf5-909f-4030-858e-2ddd33149720 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:12.378897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fecf5-909f-4030-858e-2ddd33149720 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:12.380926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fecf5-909f-4030-858e-2ddd33149720 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:12.386488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 295/579] 10.222.0.58 () {56 vars in 1124 bytes} [Fri Oct 28 18:54:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:12.388390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c6b4-5326-410c-b9db-607c1ff02fe8 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.388676 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c6b4-5326-410c-b9db-607c1ff02fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.388946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c6b4-5326-410c-b9db-607c1ff02fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.404061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:12.407790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ea97a-27a3-4e83-93d3-cf87ed73d1a9 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:12.409836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2763c6b4-5326-410c-b9db-607c1ff02fe8 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:12.412721 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2763c6b4-5326-410c-b9db-607c1ff02fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:12.447628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392596dc-942c-4f7d-8b8e-d5d528898853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:12.448569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392596dc-942c-4f7d-8b8e-d5d528898853 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.448996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392596dc-942c-4f7d-8b8e-d5d528898853 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.449474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392596dc-942c-4f7d-8b8e-d5d528898853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.450383 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392596dc-942c-4f7d-8b8e-d5d528898853 None admin] RBAC: Adding query filter params (role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=a758611cf8d94e8ca8aad6a75bef5e34) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:12.450851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392596dc-942c-4f7d-8b8e-d5d528898853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:12.452489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392596dc-942c-4f7d-8b8e-d5d528898853 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:12.467441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 285/580] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:12 2022] GET /identity/v3/role_assignments?user.id=a758611cf8d94e8ca8aad6a75bef5e34&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:12.468850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392596dc-942c-4f7d-8b8e-d5d528898853 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:12.494444 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 296/581] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:12.550245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:12.630843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392596dc-942c-4f7d-8b8e-d5d528898853 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:12.680815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf7c57-c7b0-42b3-84e6-6b46fc7c23dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:12.681449 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf7c57-c7b0-42b3-84e6-6b46fc7c23dc None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.681728 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf7c57-c7b0-42b3-84e6-6b46fc7c23dc None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.681996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf7c57-c7b0-42b3-84e6-6b46fc7c23dc None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.682708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf7c57-c7b0-42b3-84e6-6b46fc7c23dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:12.683031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf7c57-c7b0-42b3-84e6-6b46fc7c23dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:12.684477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf7c57-c7b0-42b3-84e6-6b46fc7c23dc None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:12.692253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 286/582] 10.222.0.58 () {56 vars in 1124 bytes} [Fri Oct 28 18:54:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:12.697707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c6b4-5326-410c-b9db-607c1ff02fe8 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:12.767055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f05ad-8da9-422a-95ce-92b48b1c1208 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.767342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f05ad-8da9-422a-95ce-92b48b1c1208 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.767613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f05ad-8da9-422a-95ce-92b48b1c1208 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.769467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 287/583] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:12.774401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f84e6-263b-4b98-93fa-a8f79694b58b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.774686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f84e6-263b-4b98-93fa-a8f79694b58b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.774954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f84e6-263b-4b98-93fa-a8f79694b58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.796608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-ad4f84e6-263b-4b98-93fa-a8f79694b58b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:12.797795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-ad4f84e6-263b-4b98-93fa-a8f79694b58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:12.812653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c6b4-5326-410c-b9db-607c1ff02fe8 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:12.856447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f5a5c-f3d3-4579-b35d-2ad204daa999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:12.857074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f5a5c-f3d3-4579-b35d-2ad204daa999 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:12.857355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f5a5c-f3d3-4579-b35d-2ad204daa999 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:12.857625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f5a5c-f3d3-4579-b35d-2ad204daa999 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:12.858327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f5a5c-f3d3-4579-b35d-2ad204daa999 None admin] RBAC: Adding query filter params (interface=public, service_id=38c9e6f87aac402c938c16460449ecb3, region_id=RegionOne) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:12.859211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f5a5c-f3d3-4579-b35d-2ad204daa999 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:12.860607 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f5a5c-f3d3-4579-b35d-2ad204daa999 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:12.867316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:12.869363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 297/584] 10.222.0.58 () {56 vars in 1251 bytes} [Fri Oct 28 18:54:12 2022] GET /identity/v3/endpoints?service_id=38c9e6f87aac402c938c16460449ecb3&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:12.870641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f5a5c-f3d3-4579-b35d-2ad204daa999 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:12.876076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 288/585] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:12.877038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f84e6-263b-4b98-93fa-a8f79694b58b None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:12.937703 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:13.030399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f5a5c-f3d3-4579-b35d-2ad204daa999 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.046683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f84e6-263b-4b98-93fa-a8f79694b58b None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.090601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968d4b7-1c49-4cc5-a7e8-aad5e73f96f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.091184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968d4b7-1c49-4cc5-a7e8-aad5e73f96f1 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.091470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968d4b7-1c49-4cc5-a7e8-aad5e73f96f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.091745 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968d4b7-1c49-4cc5-a7e8-aad5e73f96f1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.092330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968d4b7-1c49-4cc5-a7e8-aad5e73f96f1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:13.093707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968d4b7-1c49-4cc5-a7e8-aad5e73f96f1 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:13.094694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bb44d-ddb5-4a4d-932c-85aedcc3fcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.095326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb44d-ddb5-4a4d-932c-85aedcc3fcf3 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.095600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb44d-ddb5-4a4d-932c-85aedcc3fcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.095867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb44d-ddb5-4a4d-932c-85aedcc3fcf3 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.096469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bb44d-ddb5-4a4d-932c-85aedcc3fcf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:13.097874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bb44d-ddb5-4a4d-932c-85aedcc3fcf3 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:13.111180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 289/586] 10.222.0.58 () {56 vars in 1087 bytes} [Fri Oct 28 18:54:12 2022] GET /identity/v3/services => generated 772 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-0968d4b7-1c49-4cc5-a7e8-aad5e73f96f1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:13.111963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:13.112999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:13.113973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 298/587] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:12 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:13.119571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bb44d-ddb5-4a4d-932c-85aedcc3fcf3 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:13.207620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968d4b7-1c49-4cc5-a7e8-aad5e73f96f1 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:13.250190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:13.254721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bb44d-ddb5-4a4d-932c-85aedcc3fcf3 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.276662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:13.296744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0ef69-2206-414f-803e-7ecbe167e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.297759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0ef69-2206-414f-803e-7ecbe167e53f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.298245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0ef69-2206-414f-803e-7ecbe167e53f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.298720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0ef69-2206-414f-803e-7ecbe167e53f None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.299719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0ef69-2206-414f-803e-7ecbe167e53f None admin] RBAC: Adding query filter params (name=network) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:13.300344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0ef69-2206-414f-803e-7ecbe167e53f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:13.302024 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0ef69-2206-414f-803e-7ecbe167e53f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:13.308088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 290/588] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:13.315026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0ef69-2206-414f-803e-7ecbe167e53f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:13.317046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968d4b7-1c49-4cc5-a7e8-aad5e73f96f1 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.359027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9d440-f9a9-4300-9aef-6d24bbd9d9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.359664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9d440-f9a9-4300-9aef-6d24bbd9d9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.359915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9d440-f9a9-4300-9aef-6d24bbd9d9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.360183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9d440-f9a9-4300-9aef-6d24bbd9d9f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.378973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9d440-f9a9-4300-9aef-6d24bbd9d9f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-96a9d440-f9a9-4300-9aef-6d24bbd9d9f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:13.384520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:13.385469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:13.386302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:13.391007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 299/589] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:54:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:13.392598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9d440-f9a9-4300-9aef-6d24bbd9d9f5 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:13.471652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0ef69-2206-414f-803e-7ecbe167e53f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.505882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9d440-f9a9-4300-9aef-6d24bbd9d9f5 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.509480 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbca764-3998-4832-9b51-498fef1ea3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.510327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca764-3998-4832-9b51-498fef1ea3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.510810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca764-3998-4832-9b51-498fef1ea3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.511328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca764-3998-4832-9b51-498fef1ea3b6 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.513301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbca764-3998-4832-9b51-498fef1ea3b6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:13.514089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbca764-3998-4832-9b51-498fef1ea3b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:13.516201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbca764-3998-4832-9b51-498fef1ea3b6 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:13.522759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 291/590] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:13.529038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbca764-3998-4832-9b51-498fef1ea3b6 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:13.532454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:13.544594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e8e9-7387-4d48-af34-a3d608d5b083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.545335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e8e9-7387-4d48-af34-a3d608d5b083 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.545714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e8e9-7387-4d48-af34-a3d608d5b083 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.546137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e8e9-7387-4d48-af34-a3d608d5b083 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.546920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e8e9-7387-4d48-af34-a3d608d5b083 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:13.547532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e8e9-7387-4d48-af34-a3d608d5b083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:13.549407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e8e9-7387-4d48-af34-a3d608d5b083 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:13.562586 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:13.567113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 300/591] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:13.569722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e8e9-7387-4d48-af34-a3d608d5b083 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:13.650050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbca764-3998-4832-9b51-498fef1ea3b6 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.679432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e8e9-7387-4d48-af34-a3d608d5b083 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.688076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bb62a-2211-4668-b240-7aadbde10376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.688697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bb62a-2211-4668-b240-7aadbde10376 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.688978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bb62a-2211-4668-b240-7aadbde10376 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.689252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bb62a-2211-4668-b240-7aadbde10376 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.689954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bb62a-2211-4668-b240-7aadbde10376 None admin] RBAC: Adding query filter params (interface=public, service_id=a55e2da62ee643099047ea5e67fce6ce, region_id=RegionOne) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:13.690264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bb62a-2211-4668-b240-7aadbde10376 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:13.691754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bb62a-2211-4668-b240-7aadbde10376 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:13.700491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 292/592] 10.222.0.58 () {56 vars in 1251 bytes} [Fri Oct 28 18:54:13 2022] GET /identity/v3/endpoints?service_id=a55e2da62ee643099047ea5e67fce6ce&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:13.706090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bb62a-2211-4668-b240-7aadbde10376 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:13.717799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14ee20-ffbc-4b14-9ef1-96b187ee52de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.718864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14ee20-ffbc-4b14-9ef1-96b187ee52de None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.719369 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14ee20-ffbc-4b14-9ef1-96b187ee52de None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.719903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14ee20-ffbc-4b14-9ef1-96b187ee52de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.733303 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14ee20-ffbc-4b14-9ef1-96b187ee52de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:13.735186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14ee20-ffbc-4b14-9ef1-96b187ee52de None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-ee14ee20-ffbc-4b14-9ef1-96b187ee52de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:13.754352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:13.755408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:13.756284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:13.756284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:13.756284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:13.757640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 301/593] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:13.758683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14ee20-ffbc-4b14-9ef1-96b187ee52de None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:13.809712 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:13.843777 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:13.856431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bb62a-2211-4668-b240-7aadbde10376 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.876778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14ee20-ffbc-4b14-9ef1-96b187ee52de None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:13.901865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcc084-c6ae-4fcc-9be4-56d95443d619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.902489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcc084-c6ae-4fcc-9be4-56d95443d619 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.902935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcc084-c6ae-4fcc-9be4-56d95443d619 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.903300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcc084-c6ae-4fcc-9be4-56d95443d619 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.904193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fcc084-c6ae-4fcc-9be4-56d95443d619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:13.905526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fcc084-c6ae-4fcc-9be4-56d95443d619 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:13.919549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 293/594] 10.222.0.58 () {56 vars in 1087 bytes} [Fri Oct 28 18:54:13 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:13.919549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41005299-d6f3-4935-9db8-16098b44a7a8 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.919549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41005299-d6f3-4935-9db8-16098b44a7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.919549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41005299-d6f3-4935-9db8-16098b44a7a8 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.919549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 294/595] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:13.921838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a4c97-6f9f-466d-be37-22a1b7f09f54 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.922118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a4c97-6f9f-466d-be37-22a1b7f09f54 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.922635 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a4c97-6f9f-466d-be37-22a1b7f09f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.936714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9a321-7331-4521-b7ce-35c600e4d3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:13.937395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9a321-7331-4521-b7ce-35c600e4d3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:13.937667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9a321-7331-4521-b7ce-35c600e4d3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:13.937929 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9a321-7331-4521-b7ce-35c600e4d3d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:13.938871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9a321-7331-4521-b7ce-35c600e4d3d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:13.940805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9a321-7331-4521-b7ce-35c600e4d3d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:13.941602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9a321-7331-4521-b7ce-35c600e4d3d6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:13.960486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3b3a4c97-6f9f-466d-be37-22a1b7f09f54 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:13.961702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3b3a4c97-6f9f-466d-be37-22a1b7f09f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:13.969118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 302/596] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:13.982029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9a321-7331-4521-b7ce-35c600e4d3d6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:14.088326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 295/597] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:14.136807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:14.142530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9a321-7331-4521-b7ce-35c600e4d3d6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:14.181255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc948fe6-edf9-486f-9671-609ce4bfb439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:14.181804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc948fe6-edf9-486f-9671-609ce4bfb439 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:14.182077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc948fe6-edf9-486f-9671-609ce4bfb439 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:14.182351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc948fe6-edf9-486f-9671-609ce4bfb439 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:14.189294 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc948fe6-edf9-486f-9671-609ce4bfb439 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:14.190725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc948fe6-edf9-486f-9671-609ce4bfb439 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-cc948fe6-edf9-486f-9671-609ce4bfb439 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:14.202884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:14.203850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:14.204862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 303/598] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:54:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:14.212446 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a4c97-6f9f-466d-be37-22a1b7f09f54 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:14.316026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:14.332784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a4c97-6f9f-466d-be37-22a1b7f09f54 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:14.377344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42e643-49d3-4001-a27f-1ce7028e4569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:14.377953 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42e643-49d3-4001-a27f-1ce7028e4569 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:14.378345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42e643-49d3-4001-a27f-1ce7028e4569 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:14.378552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42e643-49d3-4001-a27f-1ce7028e4569 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:14.379361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42e643-49d3-4001-a27f-1ce7028e4569 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:14.379670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42e643-49d3-4001-a27f-1ce7028e4569 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:14.381216 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42e643-49d3-4001-a27f-1ce7028e4569 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:14.400501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 296/599] 10.222.0.58 () {56 vars in 1135 bytes} [Fri Oct 28 18:54:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:14.406756 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc948fe6-edf9-486f-9671-609ce4bfb439 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:14.436917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42e643-49d3-4001-a27f-1ce7028e4569 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:14.586090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42e643-49d3-4001-a27f-1ce7028e4569 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:14.625767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7b0f0-317c-4020-ade8-6e046032664b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:14.626542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7b0f0-317c-4020-ade8-6e046032664b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:14.626828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7b0f0-317c-4020-ade8-6e046032664b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:14.627132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7b0f0-317c-4020-ade8-6e046032664b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:14.638644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7b0f0-317c-4020-ade8-6e046032664b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-c7c7b0f0-317c-4020-ade8-6e046032664b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:14.644591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:14.645526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:14.646350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 297/600] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:54:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:14.651547 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7b0f0-317c-4020-ade8-6e046032664b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:14.663344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:14.673242 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc948fe6-edf9-486f-9671-609ce4bfb439 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:14.747101 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c87701-521d-4d88-af4f-ae6ee08a4596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:14.747694 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c87701-521d-4d88-af4f-ae6ee08a4596 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:14.747976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c87701-521d-4d88-af4f-ae6ee08a4596 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:14.748243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c87701-521d-4d88-af4f-ae6ee08a4596 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:14.802584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:14.830194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7b0f0-317c-4020-ade8-6e046032664b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:14.870779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3232f-ec19-487a-9b59-eba20071922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:14.871807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3232f-ec19-487a-9b59-eba20071922a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:14.872291 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3232f-ec19-487a-9b59-eba20071922a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:14.872764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3232f-ec19-487a-9b59-eba20071922a None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:14.873629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3232f-ec19-487a-9b59-eba20071922a None admin] RBAC: Adding query filter params (name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:14.874134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3232f-ec19-487a-9b59-eba20071922a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:14.875732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3232f-ec19-487a-9b59-eba20071922a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:14.878854 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c87701-521d-4d88-af4f-ae6ee08a4596 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:14.880341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c87701-521d-4d88-af4f-ae6ee08a4596 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:14.885448 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 298/601] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:54:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:14.890801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3232f-ec19-487a-9b59-eba20071922a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:15.036698 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:15.052219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 304/602] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:54:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:54:15.058859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c87701-521d-4d88-af4f-ae6ee08a4596 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:15.097177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3232f-ec19-487a-9b59-eba20071922a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:15.142072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22db6e1-4588-4c0e-8d9b-93ff07631909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:15.143188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db6e1-4588-4c0e-8d9b-93ff07631909 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.144026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:15.144505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db6e1-4588-4c0e-8d9b-93ff07631909 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.145678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db6e1-4588-4c0e-8d9b-93ff07631909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:15.158855 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22db6e1-4588-4c0e-8d9b-93ff07631909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:15.160625 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22db6e1-4588-4c0e-8d9b-93ff07631909 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-c22db6e1-4588-4c0e-8d9b-93ff07631909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:15.176026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:15.176995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:15.177908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:15.177908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:15.177908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:15.180158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 299/603] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:15.181651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1d34-eef0-47f3-939c-a1952e9c739d None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.182154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1d34-eef0-47f3-939c-a1952e9c739d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.182627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1d34-eef0-47f3-939c-a1952e9c739d None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:15.184340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 300/604] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:15.185451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1d34-eef0-47f3-939c-a1952e9c739d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:15.191364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c87701-521d-4d88-af4f-ae6ee08a4596 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:15.236238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d66b2a-8c7a-467c-96c5-e4644eab1587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:15.236859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d66b2a-8c7a-467c-96c5-e4644eab1587 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.237132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d66b2a-8c7a-467c-96c5-e4644eab1587 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.237395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d66b2a-8c7a-467c-96c5-e4644eab1587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:15.248978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d66b2a-8c7a-467c-96c5-e4644eab1587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:15.250524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d66b2a-8c7a-467c-96c5-e4644eab1587 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-31d66b2a-8c7a-467c-96c5-e4644eab1587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:15.267864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:15.269063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:15.269984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:15.269984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:15.269984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:15.274045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 305/605] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:15.276216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aeeba-da90-42a9-b6db-ff85cd0313b1 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.276500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aeeba-da90-42a9-b6db-ff85cd0313b1 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.276770 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aeeba-da90-42a9-b6db-ff85cd0313b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:15.298619 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-b59aeeba-da90-42a9-b6db-ff85cd0313b1 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:15.299851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b59aeeba-da90-42a9-b6db-ff85cd0313b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:15.311234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1d34-eef0-47f3-939c-a1952e9c739d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:15.333668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:15.355759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa51ee9-fbfa-43f4-8da4-b478f9264937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:15.356741 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa51ee9-fbfa-43f4-8da4-b478f9264937 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.357342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa51ee9-fbfa-43f4-8da4-b478f9264937 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.357911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa51ee9-fbfa-43f4-8da4-b478f9264937 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:15.358799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa51ee9-fbfa-43f4-8da4-b478f9264937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:15.359337 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa51ee9-fbfa-43f4-8da4-b478f9264937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:15.361174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa51ee9-fbfa-43f4-8da4-b478f9264937 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:15.375235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 306/606] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:15.376171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aeeba-da90-42a9-b6db-ff85cd0313b1 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:15.377739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 301/607] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:15.385829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa51ee9-fbfa-43f4-8da4-b478f9264937 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:15.462312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:15.541820 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa51ee9-fbfa-43f4-8da4-b478f9264937 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:15.580013 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ec6a8-7832-4eb0-a650-ce6bf2a54a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:15.581499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ec6a8-7832-4eb0-a650-ce6bf2a54a1f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.581775 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ec6a8-7832-4eb0-a650-ce6bf2a54a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.582043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ec6a8-7832-4eb0-a650-ce6bf2a54a1f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:15.586946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aeeba-da90-42a9-b6db-ff85cd0313b1 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:15.595577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ec6a8-7832-4eb0-a650-ce6bf2a54a1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:15.597020 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ec6a8-7832-4eb0-a650-ce6bf2a54a1f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-3a4ec6a8-7832-4eb0-a650-ce6bf2a54a1f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:15.618394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:15.619367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:15.620307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 302/608] 10.222.0.58 () {56 vars in 1117 bytes} [Fri Oct 28 18:54:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:15.627307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ec6a8-7832-4eb0-a650-ce6bf2a54a1f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:15.671664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b75d86-5794-4f96-a034-6f1656a1ea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:15.672273 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b75d86-5794-4f96-a034-6f1656a1ea3d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.672547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b75d86-5794-4f96-a034-6f1656a1ea3d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.672812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b75d86-5794-4f96-a034-6f1656a1ea3d None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:15.673441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b75d86-5794-4f96-a034-6f1656a1ea3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:15.673737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b75d86-5794-4f96-a034-6f1656a1ea3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:15.675140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b75d86-5794-4f96-a034-6f1656a1ea3d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:15.694412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 307/609] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:15.699263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b75d86-5794-4f96-a034-6f1656a1ea3d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:15.700221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:15.764481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ec6a8-7832-4eb0-a650-ce6bf2a54a1f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:15.817989 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc2d00-985e-4ec5-821c-6e5554968795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:15.818614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc2d00-985e-4ec5-821c-6e5554968795 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.818889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc2d00-985e-4ec5-821c-6e5554968795 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.819178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc2d00-985e-4ec5-821c-6e5554968795 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:15.819923 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc2d00-985e-4ec5-821c-6e5554968795 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:15.820209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc2d00-985e-4ec5-821c-6e5554968795 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:15.821753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc2d00-985e-4ec5-821c-6e5554968795 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:15.826074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:15.830438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b75d86-5794-4f96-a034-6f1656a1ea3d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:15.843558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 303/610] 10.222.0.58 () {56 vars in 1149 bytes} [Fri Oct 28 18:54:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:15.844498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc2d00-985e-4ec5-821c-6e5554968795 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:15.875894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ec3b4-774b-4e41-93d9-1994356c39b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:15.877410 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ec3b4-774b-4e41-93d9-1994356c39b1 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.877967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ec3b4-774b-4e41-93d9-1994356c39b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.878570 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ec3b4-774b-4e41-93d9-1994356c39b1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:15.879335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ec3b4-774b-4e41-93d9-1994356c39b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:15.880962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ec3b4-774b-4e41-93d9-1994356c39b1 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-532ec3b4-774b-4e41-93d9-1994356c39b1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:15.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:15.892574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 18:54:15.893715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:15.896680 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 308/611] 10.222.0.58 () {56 vars in 1132 bytes} [Fri Oct 28 18:54:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:15.897981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ec3b4-774b-4e41-93d9-1994356c39b1 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:15.953236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:15.959729 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc2d00-985e-4ec5-821c-6e5554968795 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:15.997234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd854ccc-9bf5-4daf-83f1-811fd2f56db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:15.997853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd854ccc-9bf5-4daf-83f1-811fd2f56db2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:15.998126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd854ccc-9bf5-4daf-83f1-811fd2f56db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:15.998394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd854ccc-9bf5-4daf-83f1-811fd2f56db2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.005414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd854ccc-9bf5-4daf-83f1-811fd2f56db2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:16.006958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd854ccc-9bf5-4daf-83f1-811fd2f56db2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:16.011063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ec3b4-774b-4e41-93d9-1994356c39b1 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-dd854ccc-9bf5-4daf-83f1-811fd2f56db2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:16.019204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:16.020325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 304/612] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:16.021222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd854ccc-9bf5-4daf-83f1-811fd2f56db2 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:16.053470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509cdce-a1e2-4ba3-b732-62b465cc200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:16.054080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509cdce-a1e2-4ba3-b732-62b465cc200e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.054352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509cdce-a1e2-4ba3-b732-62b465cc200e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.054625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509cdce-a1e2-4ba3-b732-62b465cc200e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.087602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:16.125564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd854ccc-9bf5-4daf-83f1-811fd2f56db2 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:16.129295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509cdce-a1e2-4ba3-b732-62b465cc200e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:16.130748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509cdce-a1e2-4ba3-b732-62b465cc200e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:16.163545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d30d81-6ac4-4b33-a088-a3bcc0006dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:16.164145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d30d81-6ac4-4b33-a088-a3bcc0006dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.164423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d30d81-6ac4-4b33-a088-a3bcc0006dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.164691 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d30d81-6ac4-4b33-a088-a3bcc0006dc5 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.165308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d30d81-6ac4-4b33-a088-a3bcc0006dc5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:16.165608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d30d81-6ac4-4b33-a088-a3bcc0006dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:16.166959 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d30d81-6ac4-4b33-a088-a3bcc0006dc5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:16.171526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 305/613] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:16.172744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09febdb4-ea28-4d51-ae9e-8d97d0a73d0d None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.173026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09febdb4-ea28-4d51-ae9e-8d97d0a73d0d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.173290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09febdb4-ea28-4d51-ae9e-8d97d0a73d0d None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.174625 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 306/614] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:16.179381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09febdb4-ea28-4d51-ae9e-8d97d0a73d0d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:16.202343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:16.244311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 309/615] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:54:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:54:16.244311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509cdce-a1e2-4ba3-b732-62b465cc200e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:16.290638 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09febdb4-ea28-4d51-ae9e-8d97d0a73d0d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:16.308207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:16.325676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05f19f-94c8-4b88-970e-f187d6cc905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:16.326282 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05f19f-94c8-4b88-970e-f187d6cc905d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.326558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05f19f-94c8-4b88-970e-f187d6cc905d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.326825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05f19f-94c8-4b88-970e-f187d6cc905d None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.327645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05f19f-94c8-4b88-970e-f187d6cc905d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:16.327943 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05f19f-94c8-4b88-970e-f187d6cc905d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:16.329364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05f19f-94c8-4b88-970e-f187d6cc905d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:16.342504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 307/616] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:16.343923 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5936b-921e-4392-86b4-e3266f5dc979 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.344242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5936b-921e-4392-86b4-e3266f5dc979 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.344512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5936b-921e-4392-86b4-e3266f5dc979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.359835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509cdce-a1e2-4ba3-b732-62b465cc200e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:16.363552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-cac5936b-921e-4392-86b4-e3266f5dc979 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:16.364774 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-cac5936b-921e-4392-86b4-e3266f5dc979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:16.404362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd4153-575b-4004-b2ca-6b1edc4613cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:16.405209 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd4153-575b-4004-b2ca-6b1edc4613cb None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.405654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd4153-575b-4004-b2ca-6b1edc4613cb None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.406065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd4153-575b-4004-b2ca-6b1edc4613cb None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.406820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd4153-575b-4004-b2ca-6b1edc4613cb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:16.407293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd4153-575b-4004-b2ca-6b1edc4613cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:16.409391 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd4153-575b-4004-b2ca-6b1edc4613cb None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:16.415283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 310/617] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:16.416251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd4153-575b-4004-b2ca-6b1edc4613cb None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:16.455466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 308/618] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:16.456961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5936b-921e-4392-86b4-e3266f5dc979 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:16.474697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:16.554530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd4153-575b-4004-b2ca-6b1edc4613cb None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:16.585064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5936b-921e-4392-86b4-e3266f5dc979 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:16.590830 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:16.602467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64d839-c75b-4015-8ea5-2b1b060d2306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:16.603067 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64d839-c75b-4015-8ea5-2b1b060d2306 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.603350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64d839-c75b-4015-8ea5-2b1b060d2306 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.603624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64d839-c75b-4015-8ea5-2b1b060d2306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.617146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64d839-c75b-4015-8ea5-2b1b060d2306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:16.618600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64d839-c75b-4015-8ea5-2b1b060d2306 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:16.633754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752d289-f671-47ab-ae79-4befa92bcf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:16.634355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752d289-f671-47ab-ae79-4befa92bcf4c None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.634632 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752d289-f671-47ab-ae79-4befa92bcf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.634901 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752d289-f671-47ab-ae79-4befa92bcf4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-bc64d839-c75b-4015-8ea5-2b1b060d2306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:16.635582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:16.636479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:16.637436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:16.637436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:16.637436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:16.637614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752d289-f671-47ab-ae79-4befa92bcf4c None admin] RBAC: Adding query filter params (role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=a51690d0367b45db8a6552764cc1d11e) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:16.637614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752d289-f671-47ab-ae79-4befa92bcf4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:16.637752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 311/619] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:16.638273 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137105b-9908-4266-8d8e-45a20fbb12ba None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.638273 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137105b-9908-4266-8d8e-45a20fbb12ba None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.638434 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137105b-9908-4266-8d8e-45a20fbb12ba None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.639830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752d289-f671-47ab-ae79-4befa92bcf4c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:16.639936 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 312/620] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:16.642651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137105b-9908-4266-8d8e-45a20fbb12ba None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:16.647065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 309/621] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:16 2022] GET /identity/v3/role_assignments?user.id=a51690d0367b45db8a6552764cc1d11e&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:16.648298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76dc6a-5f80-41fb-a45e-706697182c2f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.648545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76dc6a-5f80-41fb-a45e-706697182c2f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.648885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76dc6a-5f80-41fb-a45e-706697182c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.668723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6f76dc6a-5f80-41fb-a45e-706697182c2f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:16.670278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6f76dc6a-5f80-41fb-a45e-706697182c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:16.779635 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 310/622] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:16.780502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76dc6a-5f80-41fb-a45e-706697182c2f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:16.783847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:16.795412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137105b-9908-4266-8d8e-45a20fbb12ba None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:16.839921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12b1e-9229-4add-b16e-e068645d5fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:16.840687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12b1e-9229-4add-b16e-e068645d5fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.841443 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12b1e-9229-4add-b16e-e068645d5fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.842338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12b1e-9229-4add-b16e-e068645d5fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.843439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a12b1e-9229-4add-b16e-e068645d5fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:16.843874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a12b1e-9229-4add-b16e-e068645d5fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:16.845552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a12b1e-9229-4add-b16e-e068645d5fd1 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:16.861873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 313/623] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:16.868380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12b1e-9229-4add-b16e-e068645d5fd1 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:16.903312 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76dc6a-5f80-41fb-a45e-706697182c2f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:16.926306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:16.958033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a3193-8763-43f9-ba3e-f81979940d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:16.959294 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a3193-8763-43f9-ba3e-f81979940d86 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:16.959805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a3193-8763-43f9-ba3e-f81979940d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:16.960860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a3193-8763-43f9-ba3e-f81979940d86 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:16.961909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a3193-8763-43f9-ba3e-f81979940d86 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:16.963824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a3193-8763-43f9-ba3e-f81979940d86 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-849a3193-8763-43f9-ba3e-f81979940d86 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:16.977761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:16.978790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 18:54:16.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:16.981874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 311/624] 10.222.0.58 () {56 vars in 1129 bytes} [Fri Oct 28 18:54:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:16.986946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a3193-8763-43f9-ba3e-f81979940d86 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:17.009897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12b1e-9229-4add-b16e-e068645d5fd1 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:17.057666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a4869-bfb4-4360-8a59-e9cbd4eee79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:17.058306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a4869-bfb4-4360-8a59-e9cbd4eee79b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:17.058618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a4869-bfb4-4360-8a59-e9cbd4eee79b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:17.058888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a4869-bfb4-4360-8a59-e9cbd4eee79b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:17.067468 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a4869-bfb4-4360-8a59-e9cbd4eee79b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:17.069191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a4869-bfb4-4360-8a59-e9cbd4eee79b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-f35a4869-bfb4-4360-8a59-e9cbd4eee79b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:17.085768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:17.086682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:17.087587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:17.087587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:17.087587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:17.087587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:17.087587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:17.087587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:17.087587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:17.087587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:17.093739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 314/625] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:17.094264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:17.094587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a4869-bfb4-4360-8a59-e9cbd4eee79b None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:17.133253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a3193-8763-43f9-ba3e-f81979940d86 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:17.169624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa308995-3c61-4803-8a49-6bddc35e1d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:17.170407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa308995-3c61-4803-8a49-6bddc35e1d1c None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:17.170686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa308995-3c61-4803-8a49-6bddc35e1d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:17.171015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa308995-3c61-4803-8a49-6bddc35e1d1c None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:17.171652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa308995-3c61-4803-8a49-6bddc35e1d1c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:17.171952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa308995-3c61-4803-8a49-6bddc35e1d1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:17.173321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa308995-3c61-4803-8a49-6bddc35e1d1c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:17.178277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 312/626] 10.222.0.58 () {56 vars in 1124 bytes} [Fri Oct 28 18:54:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:17.179156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa308995-3c61-4803-8a49-6bddc35e1d1c None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:17.205531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:17.209690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a4869-bfb4-4360-8a59-e9cbd4eee79b None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:17.247071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599cd64-905f-45b1-8a88-e1d080715fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:17.248118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599cd64-905f-45b1-8a88-e1d080715fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:17.248566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599cd64-905f-45b1-8a88-e1d080715fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:17.249000 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599cd64-905f-45b1-8a88-e1d080715fc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:17.249850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599cd64-905f-45b1-8a88-e1d080715fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:17.250331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599cd64-905f-45b1-8a88-e1d080715fc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:17.252184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599cd64-905f-45b1-8a88-e1d080715fc8 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:17.271101 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 315/627] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:17.272460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599cd64-905f-45b1-8a88-e1d080715fc8 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:17.283890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa308995-3c61-4803-8a49-6bddc35e1d1c None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:17.314667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:17.320252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cda015-180b-4982-ba9f-bd6420fc73ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:17.320886 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cda015-180b-4982-ba9f-bd6420fc73ef None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:17.321471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cda015-180b-4982-ba9f-bd6420fc73ef None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:17.321996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cda015-180b-4982-ba9f-bd6420fc73ef None admin] PATH_INFO: `/v3/services/network` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:17.322912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cda015-180b-4982-ba9f-bd6420fc73ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:17.325018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cda015-180b-4982-ba9f-bd6420fc73ef None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-76cda015-180b-4982-ba9f-bd6420fc73ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:17.334526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:17.335757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:17.336738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 313/628] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:17 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:17.337765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cda015-180b-4982-ba9f-bd6420fc73ef None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:17.394896 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599cd64-905f-45b1-8a88-e1d080715fc8 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:17.430907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e39cd9-2ac1-4489-9d68-818b0ca71b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:17.431548 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e39cd9-2ac1-4489-9d68-818b0ca71b2e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:17.431830 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e39cd9-2ac1-4489-9d68-818b0ca71b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:17.432104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e39cd9-2ac1-4489-9d68-818b0ca71b2e None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:17.432738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e39cd9-2ac1-4489-9d68-818b0ca71b2e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:17.434195 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e39cd9-2ac1-4489-9d68-818b0ca71b2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:17.435950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e39cd9-2ac1-4489-9d68-818b0ca71b2e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:17.442445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 316/629] 10.222.0.58 () {56 vars in 1124 bytes} [Fri Oct 28 18:54:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:17.443698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e39cd9-2ac1-4489-9d68-818b0ca71b2e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:17.446288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cda015-180b-4982-ba9f-bd6420fc73ef None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:17.477626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:17.484351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5b756-3e45-4348-a7fc-b8ca354500ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:17.485080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5b756-3e45-4348-a7fc-b8ca354500ad None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:17.485426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5b756-3e45-4348-a7fc-b8ca354500ad None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:17.485771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5b756-3e45-4348-a7fc-b8ca354500ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:17.486581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5b756-3e45-4348-a7fc-b8ca354500ad None admin] RBAC: Adding query filter params (role.id=01b71dd1f22e41269c3caa83dea57b2e, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=0eef740ccf884c079e6a2d6cdc89f4bb) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:17.487053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5b756-3e45-4348-a7fc-b8ca354500ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:17.488664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5b756-3e45-4348-a7fc-b8ca354500ad None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:17.495865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 314/630] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:17 2022] GET /identity/v3/role_assignments?user.id=0eef740ccf884c079e6a2d6cdc89f4bb&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=01b71dd1f22e41269c3caa83dea57b2e => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:17.497434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5b756-3e45-4348-a7fc-b8ca354500ad None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:17.611719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:17.629388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e39cd9-2ac1-4489-9d68-818b0ca71b2e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:17.689389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5b756-3e45-4348-a7fc-b8ca354500ad None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:17.700104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15fa89-94a8-4fbb-8ceb-f588ffe07bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:17.700726 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15fa89-94a8-4fbb-8ceb-f588ffe07bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:17.701002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15fa89-94a8-4fbb-8ceb-f588ffe07bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:17.701278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15fa89-94a8-4fbb-8ceb-f588ffe07bf5 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:17.701910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15fa89-94a8-4fbb-8ceb-f588ffe07bf5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:17.702264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15fa89-94a8-4fbb-8ceb-f588ffe07bf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:17.710119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15fa89-94a8-4fbb-8ceb-f588ffe07bf5 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:17.714757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 317/631] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:17.719924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15fa89-94a8-4fbb-8ceb-f588ffe07bf5 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:17.758318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c1ba5-d288-4581-9b88-5cc5a31d4057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:17.759461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c1ba5-d288-4581-9b88-5cc5a31d4057 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:17.759971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c1ba5-d288-4581-9b88-5cc5a31d4057 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:17.760453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c1ba5-d288-4581-9b88-5cc5a31d4057 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:17.761588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c1ba5-d288-4581-9b88-5cc5a31d4057 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:17.763373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c1ba5-d288-4581-9b88-5cc5a31d4057 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:17.797270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 315/632] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 18:54:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:17.831453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:17.845248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15fa89-94a8-4fbb-8ceb-f588ffe07bf5 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:17.889934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ca92d-c3d5-467c-a656-812e19f25479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:17.890611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ca92d-c3d5-467c-a656-812e19f25479 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:17.890887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ca92d-c3d5-467c-a656-812e19f25479 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:17.891199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ca92d-c3d5-467c-a656-812e19f25479 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:17.891814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ca92d-c3d5-467c-a656-812e19f25479 None admin] RBAC: Adding query filter params (type=network) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:17.892173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ca92d-c3d5-467c-a656-812e19f25479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:17.893838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ca92d-c3d5-467c-a656-812e19f25479 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:17.899926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 318/633] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:17.905910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c1ba5-d288-4581-9b88-5cc5a31d4057 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:18.038420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c1ba5-d288-4581-9b88-5cc5a31d4057 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:18.055535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:18.076104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d179a-b58a-4112-bdb2-64f080f0930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:18.076702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d179a-b58a-4112-bdb2-64f080f0930b None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:18.076990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d179a-b58a-4112-bdb2-64f080f0930b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:18.077270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d179a-b58a-4112-bdb2-64f080f0930b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:18.078187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d179a-b58a-4112-bdb2-64f080f0930b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:18.079614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d179a-b58a-4112-bdb2-64f080f0930b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:18.104815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 316/634] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 18:54:17 2022] POST /identity/v3/endpoints => generated 337 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:18.926529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c0a89-b572-4877-ada5-5c75ff2145b4 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:18.927084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c0a89-b572-4877-ada5-5c75ff2145b4 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:18.927084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c0a89-b572-4877-ada5-5c75ff2145b4 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:18.928693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 319/635] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:18.934506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56e324-0890-4e85-b84b-56d8eb229c86 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:18.934915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56e324-0890-4e85-b84b-56d8eb229c86 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:18.935077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56e324-0890-4e85-b84b-56d8eb229c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:18.953182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-da56e324-0890-4e85-b84b-56d8eb229c86 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:18.954368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-da56e324-0890-4e85-b84b-56d8eb229c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:19.039114 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 317/636] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:19.298899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c0a89-b572-4877-ada5-5c75ff2145b4 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:19.351200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc0f47-3597-4667-9724-3f251e007362 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:19.351542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc0f47-3597-4667-9724-3f251e007362 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:19.351760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc0f47-3597-4667-9724-3f251e007362 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:19.353345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 318/637] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:19.359343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8b114-42d0-422d-bf59-6aaacb537515 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:19.359626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8b114-42d0-422d-bf59-6aaacb537515 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:19.359899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8b114-42d0-422d-bf59-6aaacb537515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:19.379511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-fcb8b114-42d0-422d-bf59-6aaacb537515 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:19.380719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-fcb8b114-42d0-422d-bf59-6aaacb537515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:19.406228 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c0a89-b572-4877-ada5-5c75ff2145b4 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:19.441392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafcfb7-47e6-48c3-b21a-733f89432e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:19.441992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafcfb7-47e6-48c3-b21a-733f89432e24 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:19.442263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafcfb7-47e6-48c3-b21a-733f89432e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:19.442527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafcfb7-47e6-48c3-b21a-733f89432e24 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:19.443242 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafcfb7-47e6-48c3-b21a-733f89432e24 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:19.444585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafcfb7-47e6-48c3-b21a-733f89432e24 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:19.459257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 319/638] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:19.467180 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 320/639] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:54:19 2022] POST /identity/v3/services => generated 269 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:19.729580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8b114-42d0-422d-bf59-6aaacb537515 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:19.757370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:19.832275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8b114-42d0-422d-bf59-6aaacb537515 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:19.874415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7ce45-b189-4cef-b45f-899d7e662d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:19.875086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ce45-b189-4cef-b45f-899d7e662d6d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:19.875362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ce45-b189-4cef-b45f-899d7e662d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:19.875629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ce45-b189-4cef-b45f-899d7e662d6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:19.876221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7ce45-b189-4cef-b45f-899d7e662d6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:19.877699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7ce45-b189-4cef-b45f-899d7e662d6d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-0bd7ce45-b189-4cef-b45f-899d7e662d6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:19.888063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:19.889189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:19.890103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 320/640] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:19.895769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafcfb7-47e6-48c3-b21a-733f89432e24 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:19.920967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:19.999514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a73526-35e4-4213-a3d4-846233cecb82 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.000453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a73526-35e4-4213-a3d4-846233cecb82 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.000952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a73526-35e4-4213-a3d4-846233cecb82 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.002727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 321/641] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:54:20.007755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b5192-8742-4985-929f-9af563246afa None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.008293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b5192-8742-4985-929f-9af563246afa None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.008544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafcfb7-47e6-48c3-b21a-733f89432e24 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:20.009286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b5192-8742-4985-929f-9af563246afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.028412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-045b5192-8742-4985-929f-9af563246afa None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:20.029938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-045b5192-8742-4985-929f-9af563246afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:20.051261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01ba16-c956-4a10-b567-d8b0aae3c20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:20.053085 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01ba16-c956-4a10-b567-d8b0aae3c20b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.053363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01ba16-c956-4a10-b567-d8b0aae3c20b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.053633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01ba16-c956-4a10-b567-d8b0aae3c20b None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.054264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01ba16-c956-4a10-b567-d8b0aae3c20b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:20.054564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01ba16-c956-4a10-b567-d8b0aae3c20b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:20.055964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01ba16-c956-4a10-b567-d8b0aae3c20b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:20.063015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 321/642] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:20.067478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01ba16-c956-4a10-b567-d8b0aae3c20b None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:20.123015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 322/643] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:20 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:20.155414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:20.179395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01ba16-c956-4a10-b567-d8b0aae3c20b None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:20.219216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0446e-511b-4e3a-8bfd-59c902de1585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:20.219823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0446e-511b-4e3a-8bfd-59c902de1585 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.220097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0446e-511b-4e3a-8bfd-59c902de1585 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.220370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0446e-511b-4e3a-8bfd-59c902de1585 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.220996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0446e-511b-4e3a-8bfd-59c902de1585 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:20.221295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0446e-511b-4e3a-8bfd-59c902de1585 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:20.222651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0446e-511b-4e3a-8bfd-59c902de1585 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:20.231604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 322/644] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:20.292900 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2701af-fc45-4c21-8cbb-53813924ec26 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.293255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2701af-fc45-4c21-8cbb-53813924ec26 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.293436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2701af-fc45-4c21-8cbb-53813924ec26 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.295075 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 323/645] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:20.300652 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad30412-cad2-4c29-9a9e-53da7ee342e6 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.301141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad30412-cad2-4c29-9a9e-53da7ee342e6 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.301566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad30412-cad2-4c29-9a9e-53da7ee342e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.321653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6ad30412-cad2-4c29-9a9e-53da7ee342e6 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:20.323140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6ad30412-cad2-4c29-9a9e-53da7ee342e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:20.397976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2701af-fc45-4c21-8cbb-53813924ec26 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:20.400646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 323/646] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:20 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:20.427246 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:20.515601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2701af-fc45-4c21-8cbb-53813924ec26 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:20.551734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96d35-f446-407e-9c1b-a2c7ef794a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:20.552327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96d35-f446-407e-9c1b-a2c7ef794a2d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.552608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96d35-f446-407e-9c1b-a2c7ef794a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.552873 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96d35-f446-407e-9c1b-a2c7ef794a2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.564134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f96d35-f446-407e-9c1b-a2c7ef794a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-25f96d35-f446-407e-9c1b-a2c7ef794a2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:20.569060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:20.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:20.571261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 324/647] 10.222.0.58 () {56 vars in 1120 bytes} [Fri Oct 28 18:54:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:20.576685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad30412-cad2-4c29-9a9e-53da7ee342e6 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:20.645814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:20.673590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96d35-f446-407e-9c1b-a2c7ef794a2d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:20.681926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad30412-cad2-4c29-9a9e-53da7ee342e6 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:20.719377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fafdb-5009-4610-9c10-54c40bde7ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:20.720466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fafdb-5009-4610-9c10-54c40bde7ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.720812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fafdb-5009-4610-9c10-54c40bde7ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.721152 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fafdb-5009-4610-9c10-54c40bde7ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.721904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fafdb-5009-4610-9c10-54c40bde7ab1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:20.722275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fafdb-5009-4610-9c10-54c40bde7ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:20.723739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fafdb-5009-4610-9c10-54c40bde7ab1 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:20.733425 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 324/648] 10.222.0.58 () {56 vars in 1115 bytes} [Fri Oct 28 18:54:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:20.747012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fafdb-5009-4610-9c10-54c40bde7ab1 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:20.770943 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:20.795185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96d35-f446-407e-9c1b-a2c7ef794a2d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:20.831113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4853538-5ae1-4a8a-8052-c97f400d626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:20.831299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4853538-5ae1-4a8a-8052-c97f400d626b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.831531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4853538-5ae1-4a8a-8052-c97f400d626b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.831787 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4853538-5ae1-4a8a-8052-c97f400d626b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.832370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4853538-5ae1-4a8a-8052-c97f400d626b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:20.833739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4853538-5ae1-4a8a-8052-c97f400d626b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-f4853538-5ae1-4a8a-8052-c97f400d626b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:20.843376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:20.844398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 18:54:20.845211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:20.849652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 325/649] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:54:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:20.850532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4853538-5ae1-4a8a-8052-c97f400d626b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:20.876948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fafdb-5009-4610-9c10-54c40bde7ab1 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:20.908583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:20.917766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4edea0-beea-4242-a2dd-a2200dbfb5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:20.918909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4edea0-beea-4242-a2dd-a2200dbfb5df None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:20.919252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4edea0-beea-4242-a2dd-a2200dbfb5df None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:20.919797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4edea0-beea-4242-a2dd-a2200dbfb5df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:20.932338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4edea0-beea-4242-a2dd-a2200dbfb5df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:20.933831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4edea0-beea-4242-a2dd-a2200dbfb5df None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-5d4edea0-beea-4242-a2dd-a2200dbfb5df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:20.952975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:20.953934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:20.955251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:20.955251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:20.955251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:20.955251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 325/650] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:20.963903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4edea0-beea-4242-a2dd-a2200dbfb5df None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:20.989156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4853538-5ae1-4a8a-8052-c97f400d626b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:21.038346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e4904-61d1-4444-8c4e-d862a744232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:21.039004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e4904-61d1-4444-8c4e-d862a744232a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:21.039504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e4904-61d1-4444-8c4e-d862a744232a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:21.039952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e4904-61d1-4444-8c4e-d862a744232a None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:21.040652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e4904-61d1-4444-8c4e-d862a744232a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:21.041021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e4904-61d1-4444-8c4e-d862a744232a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:21.042463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e4904-61d1-4444-8c4e-d862a744232a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:21.050051 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 326/651] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:54:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:21.055731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e4904-61d1-4444-8c4e-d862a744232a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:21.085007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4edea0-beea-4242-a2dd-a2200dbfb5df None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:21.087996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:21.126478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25f849-cb2e-400c-ae4d-239d8c06235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:21.127393 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25f849-cb2e-400c-ae4d-239d8c06235a None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:21.127679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25f849-cb2e-400c-ae4d-239d8c06235a None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:21.127952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25f849-cb2e-400c-ae4d-239d8c06235a None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:21.128594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25f849-cb2e-400c-ae4d-239d8c06235a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:21.128898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25f849-cb2e-400c-ae4d-239d8c06235a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:21.130499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25f849-cb2e-400c-ae4d-239d8c06235a None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:21.145787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 326/652] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:21.150875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25f849-cb2e-400c-ae4d-239d8c06235a None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:21.186007 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e4904-61d1-4444-8c4e-d862a744232a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:21.208698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:21.225815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300c239-98da-42b3-bd44-1c7a7dd01935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:21.226231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300c239-98da-42b3-bd44-1c7a7dd01935 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:21.226509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300c239-98da-42b3-bd44-1c7a7dd01935 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:21.226782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300c239-98da-42b3-bd44-1c7a7dd01935 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:21.227474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300c239-98da-42b3-bd44-1c7a7dd01935 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:21.227773 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300c239-98da-42b3-bd44-1c7a7dd01935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:21.229125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300c239-98da-42b3-bd44-1c7a7dd01935 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:21.234493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 327/653] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:54:21 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:21.281815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25f849-cb2e-400c-ae4d-239d8c06235a None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:21.320809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6e7e3-d8c0-4f4d-b4f0-71b66441e2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:21.321736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6e7e3-d8c0-4f4d-b4f0-71b66441e2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:21.322259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6e7e3-d8c0-4f4d-b4f0-71b66441e2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:21.322684 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6e7e3-d8c0-4f4d-b4f0-71b66441e2e4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:21.331366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6e7e3-d8c0-4f4d-b4f0-71b66441e2e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:21.333114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6e7e3-d8c0-4f4d-b4f0-71b66441e2e4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-31d6e7e3-d8c0-4f4d-b4f0-71b66441e2e4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:21.346006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:21.347183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 18:54:21.348129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:21.349084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 327/654] 10.222.0.58 () {56 vars in 1117 bytes} [Fri Oct 28 18:54:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:21.355817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300c239-98da-42b3-bd44-1c7a7dd01935 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:21.451382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:21.470188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300c239-98da-42b3-bd44-1c7a7dd01935 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:21.506361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4877002-c466-45a0-8547-db7746ee3e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:21.507332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4877002-c466-45a0-8547-db7746ee3e63 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:21.507830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4877002-c466-45a0-8547-db7746ee3e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:21.508293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4877002-c466-45a0-8547-db7746ee3e63 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:21.509201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4877002-c466-45a0-8547-db7746ee3e63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:21.509697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4877002-c466-45a0-8547-db7746ee3e63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:21.511357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4877002-c466-45a0-8547-db7746ee3e63 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:21.530519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 328/655] 10.222.0.58 () {56 vars in 1149 bytes} [Fri Oct 28 18:54:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:21.536053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6e7e3-d8c0-4f4d-b4f0-71b66441e2e4 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:21.640612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:21.644882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6e7e3-d8c0-4f4d-b4f0-71b66441e2e4 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:21.682109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a9b92-ca18-41ee-a0d4-8313a0cdd833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:21.682734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a9b92-ca18-41ee-a0d4-8313a0cdd833 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:21.683025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a9b92-ca18-41ee-a0d4-8313a0cdd833 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:21.683316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a9b92-ca18-41ee-a0d4-8313a0cdd833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:21.693929 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a9b92-ca18-41ee-a0d4-8313a0cdd833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:21.695379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a9b92-ca18-41ee-a0d4-8313a0cdd833 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-753a9b92-ca18-41ee-a0d4-8313a0cdd833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:21.709560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:21.710550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:21.711494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:21.711494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:21.711494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:21.711494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 328/656] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:21.718626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4877002-c466-45a0-8547-db7746ee3e63 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:21.836145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4877002-c466-45a0-8547-db7746ee3e63 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:21.864679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:21.873187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613de04e-a38c-4b30-83cc-be200dcd135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:21.873844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613de04e-a38c-4b30-83cc-be200dcd135a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:21.874118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613de04e-a38c-4b30-83cc-be200dcd135a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:21.874387 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613de04e-a38c-4b30-83cc-be200dcd135a None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:21.875099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613de04e-a38c-4b30-83cc-be200dcd135a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:21.875380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613de04e-a38c-4b30-83cc-be200dcd135a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:21.878058 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76bd22-1df7-4041-8857-ac2e90fb5361 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:21.878335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76bd22-1df7-4041-8857-ac2e90fb5361 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:21.878598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76bd22-1df7-4041-8857-ac2e90fb5361 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:21.879273 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613de04e-a38c-4b30-83cc-be200dcd135a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:21.880217 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 329/657] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:21.891565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0447bb-dac5-4ac6-9487-989a90e3617e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:21.891908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0447bb-dac5-4ac6-9487-989a90e3617e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:21.892278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0447bb-dac5-4ac6-9487-989a90e3617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:21.895465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 329/658] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:21.901334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613de04e-a38c-4b30-83cc-be200dcd135a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:21.917765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2e0447bb-dac5-4ac6-9487-989a90e3617e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:21.919479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2e0447bb-dac5-4ac6-9487-989a90e3617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:22.012870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613de04e-a38c-4b30-83cc-be200dcd135a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:22.015071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 330/659] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:21 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:22.051806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7c288-eda1-427e-b3f2-cf96a60a7b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:22.053201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7c288-eda1-427e-b3f2-cf96a60a7b8a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:22.053684 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7c288-eda1-427e-b3f2-cf96a60a7b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:22.054202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7c288-eda1-427e-b3f2-cf96a60a7b8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:22.061550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7c288-eda1-427e-b3f2-cf96a60a7b8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:22.063315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7c288-eda1-427e-b3f2-cf96a60a7b8a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-41b7c288-eda1-427e-b3f2-cf96a60a7b8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:22.075697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:22.076605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:22.077482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:22.077482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:22.077482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:22.077482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:22.077482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:22.077482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:22.077482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:22.077482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:22.079944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 330/660] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:22.085898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0447bb-dac5-4ac6-9487-989a90e3617e None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:22.178733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:22.193710 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0447bb-dac5-4ac6-9487-989a90e3617e None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:22.228431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497863e-8812-4b90-a191-6b9b0e1cf3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:22.229028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497863e-8812-4b90-a191-6b9b0e1cf3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:22.229301 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497863e-8812-4b90-a191-6b9b0e1cf3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:22.229575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497863e-8812-4b90-a191-6b9b0e1cf3d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:22.230274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497863e-8812-4b90-a191-6b9b0e1cf3d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:22.230574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497863e-8812-4b90-a191-6b9b0e1cf3d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:22.232072 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497863e-8812-4b90-a191-6b9b0e1cf3d4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:22.243705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 331/661] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:22.249987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7c288-eda1-427e-b3f2-cf96a60a7b8a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:22.302041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:22.330971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497863e-8812-4b90-a191-6b9b0e1cf3d4 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:22.363983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7c288-eda1-427e-b3f2-cf96a60a7b8a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:22.404583 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1335ab9-ab8f-487f-af83-bf0ca498c588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:22.405723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1335ab9-ab8f-487f-af83-bf0ca498c588 None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:22.406309 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1335ab9-ab8f-487f-af83-bf0ca498c588 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:22.406803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1335ab9-ab8f-487f-af83-bf0ca498c588 None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/0eef740ccf884c079e6a2d6cdc89f4bb/roles/2604cfbe6860499e9e894574ea9c0665` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:22.409298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1335ab9-ab8f-487f-af83-bf0ca498c588 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=0eef740ccf884c079e6a2d6cdc89f4bb, role_id=2604cfbe6860499e9e894574ea9c0665)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:22.411052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1335ab9-ab8f-487f-af83-bf0ca498c588 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:22.428694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 331/662] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:54:22 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/0eef740ccf884c079e6a2d6cdc89f4bb/roles/2604cfbe6860499e9e894574ea9c0665 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:54:22.449723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497863e-8812-4b90-a191-6b9b0e1cf3d4 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:22.464964 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8f137-f0e4-4ea2-9cc9-dfd0670fdfc3 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:22.465270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8f137-f0e4-4ea2-9cc9-dfd0670fdfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:22.465518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8f137-f0e4-4ea2-9cc9-dfd0670fdfc3 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:22.467192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 332/663] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:54:22.472603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae2d92-6fbf-4d8b-a8ff-26dd2b91edf4 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:22.472993 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae2d92-6fbf-4d8b-a8ff-26dd2b91edf4 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:22.473265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae2d92-6fbf-4d8b-a8ff-26dd2b91edf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:22.488345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d3eed-a0f7-4c41-b026-b5361fe1e010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:22.489001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d3eed-a0f7-4c41-b026-b5361fe1e010 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:22.489279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d3eed-a0f7-4c41-b026-b5361fe1e010 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:22.489545 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d3eed-a0f7-4c41-b026-b5361fe1e010 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:22.490140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d3eed-a0f7-4c41-b026-b5361fe1e010 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:22.491579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d3eed-a0f7-4c41-b026-b5361fe1e010 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-340d3eed-a0f7-4c41-b026-b5361fe1e010 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:22.501907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:22.503068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 18:54:22.503958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:22.508317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 332/664] 10.222.0.58 () {56 vars in 1132 bytes} [Fri Oct 28 18:54:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:22.508796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-57ae2d92-6fbf-4d8b-a8ff-26dd2b91edf4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:22.509677 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d3eed-a0f7-4c41-b026-b5361fe1e010 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:22.510751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-57ae2d92-6fbf-4d8b-a8ff-26dd2b91edf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:22.523502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:22.611005 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 333/665] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:22 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:22.632629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d3eed-a0f7-4c41-b026-b5361fe1e010 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:22.673504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe2152-d713-4830-9775-bbc247b92b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:22.673504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe2152-d713-4830-9775-bbc247b92b97 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:22.673504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe2152-d713-4830-9775-bbc247b92b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:22.673846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe2152-d713-4830-9775-bbc247b92b97 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:22.676440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe2152-d713-4830-9775-bbc247b92b97 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:22.676440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe2152-d713-4830-9775-bbc247b92b97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:22.676440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe2152-d713-4830-9775-bbc247b92b97 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:22.680805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 333/666] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:22.686943 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae2d92-6fbf-4d8b-a8ff-26dd2b91edf4 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:22.726637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:22.788379 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae2d92-6fbf-4d8b-a8ff-26dd2b91edf4 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:22.823373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc8051-54b5-42ee-933a-4d5ad5c3680b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:22.824007 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc8051-54b5-42ee-933a-4d5ad5c3680b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:22.824279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc8051-54b5-42ee-933a-4d5ad5c3680b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:22.824547 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc8051-54b5-42ee-933a-4d5ad5c3680b None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:22.825171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc8051-54b5-42ee-933a-4d5ad5c3680b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:22.825524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc8051-54b5-42ee-933a-4d5ad5c3680b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:22.826883 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc8051-54b5-42ee-933a-4d5ad5c3680b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:22.835579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 334/667] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:22.837943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe2152-d713-4830-9775-bbc247b92b97 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:22.894255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc8051-54b5-42ee-933a-4d5ad5c3680b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:22.910763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:22.943674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe2152-d713-4830-9775-bbc247b92b97 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:22.978284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14362b-f182-4539-8ada-80310566d453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:22.978812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14362b-f182-4539-8ada-80310566d453 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:22.979124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14362b-f182-4539-8ada-80310566d453 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:22.979397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14362b-f182-4539-8ada-80310566d453 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:22.980812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14362b-f182-4539-8ada-80310566d453 None admin] RBAC: Adding query filter params (interface=public, service_id=714e17bdf8ab4f18b8d67394fba0b00a, region_id=RegionOne) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:22.981128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14362b-f182-4539-8ada-80310566d453 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:22.982490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14362b-f182-4539-8ada-80310566d453 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:22.992559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 334/668] 10.222.0.58 () {56 vars in 1251 bytes} [Fri Oct 28 18:54:22 2022] GET /identity/v3/endpoints?service_id=714e17bdf8ab4f18b8d67394fba0b00a&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:22.994577 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14362b-f182-4539-8ada-80310566d453 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:23.012462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc8051-54b5-42ee-933a-4d5ad5c3680b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:23.015351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:23.054250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4af983-f209-4ef2-b5ca-0762f797097a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:23.054887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4af983-f209-4ef2-b5ca-0762f797097a None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:23.055272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4af983-f209-4ef2-b5ca-0762f797097a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:23.055610 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4af983-f209-4ef2-b5ca-0762f797097a None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:23.056389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4af983-f209-4ef2-b5ca-0762f797097a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:23.057741 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4af983-f209-4ef2-b5ca-0762f797097a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:23.083039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 335/669] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:54:22 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:54:23.123287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14362b-f182-4539-8ada-80310566d453 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:23.162919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:23.170183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b05fc2-4b34-440f-892f-6521c6622be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:23.170869 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b05fc2-4b34-440f-892f-6521c6622be2 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:23.171180 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b05fc2-4b34-440f-892f-6521c6622be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:23.171451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b05fc2-4b34-440f-892f-6521c6622be2 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:23.172060 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b05fc2-4b34-440f-892f-6521c6622be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:23.173707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b05fc2-4b34-440f-892f-6521c6622be2 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:23.184701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 335/670] 10.222.0.58 () {56 vars in 1087 bytes} [Fri Oct 28 18:54:22 2022] GET /identity/v3/services => generated 1258 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 18:54:23.477804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35cfa2-81cb-44b7-aae9-9310f288171f None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:23.478188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35cfa2-81cb-44b7-aae9-9310f288171f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:23.478399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35cfa2-81cb-44b7-aae9-9310f288171f None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:23.479833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 336/671] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:23.485366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be1484-80f8-4498-aba9-1cebd24a5fa2 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:23.485907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be1484-80f8-4498-aba9-1cebd24a5fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:23.486401 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be1484-80f8-4498-aba9-1cebd24a5fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:23.504920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-c7be1484-80f8-4498-aba9-1cebd24a5fa2 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:23.506432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c7be1484-80f8-4498-aba9-1cebd24a5fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:23.575381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 336/672] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:23 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:23.850513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35cfa2-81cb-44b7-aae9-9310f288171f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:23.961825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35cfa2-81cb-44b7-aae9-9310f288171f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:23.996586 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129d91a-8bb9-49b0-b472-342cb3e1f503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:23.997196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129d91a-8bb9-49b0-b472-342cb3e1f503 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:23.997477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129d91a-8bb9-49b0-b472-342cb3e1f503 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:23.997751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129d91a-8bb9-49b0-b472-342cb3e1f503 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:23.998411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129d91a-8bb9-49b0-b472-342cb3e1f503 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:23.999814 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129d91a-8bb9-49b0-b472-342cb3e1f503 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:24.020592 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 337/673] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:54:23 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:24.715860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4e61f-948c-4b3b-b80a-736574db7c01 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:24.716381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4e61f-948c-4b3b-b80a-736574db7c01 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:24.716381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4e61f-948c-4b3b-b80a-736574db7c01 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:24.717953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 337/674] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:24.723384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a804e-4706-42ae-a311-d12e893b691e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:24.723796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a804e-4706-42ae-a311-d12e893b691e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:24.724000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a804e-4706-42ae-a311-d12e893b691e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:24.744613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-021a804e-4706-42ae-a311-d12e893b691e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:24.752358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-021a804e-4706-42ae-a311-d12e893b691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:24.771388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:24.829796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 338/675] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:25.095675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4e61f-948c-4b3b-b80a-736574db7c01 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:25.202314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4e61f-948c-4b3b-b80a-736574db7c01 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:25.235334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:25.239594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa3e05-d67a-4469-8fff-e866898b100d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:25.240246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa3e05-d67a-4469-8fff-e866898b100d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.240519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa3e05-d67a-4469-8fff-e866898b100d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.240788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa3e05-d67a-4469-8fff-e866898b100d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.254122 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa3e05-d67a-4469-8fff-e866898b100d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-cafa3e05-d67a-4469-8fff-e866898b100d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:25.259189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:25.260176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:25.261005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 338/676] 10.222.0.58 () {56 vars in 1120 bytes} [Fri Oct 28 18:54:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:25.266959 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a804e-4706-42ae-a311-d12e893b691e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:25.362377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebf7db-27aa-4ef2-bf54-cf3782d3267c None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.362671 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebf7db-27aa-4ef2-bf54-cf3782d3267c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.362871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebf7db-27aa-4ef2-bf54-cf3782d3267c None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.365644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 339/677] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:25.371098 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dafe3-4232-48a1-ab6c-c756b91dfb81 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.371377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dafe3-4232-48a1-ab6c-c756b91dfb81 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.371647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dafe3-4232-48a1-ab6c-c756b91dfb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.376076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a804e-4706-42ae-a311-d12e893b691e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:25.390790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-720dafe3-4232-48a1-ab6c-c756b91dfb81 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:25.398050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-720dafe3-4232-48a1-ab6c-c756b91dfb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:25.412081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c59852-930d-445b-a8c0-5aa2c68c0cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:25.412779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c59852-930d-445b-a8c0-5aa2c68c0cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.413052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c59852-930d-445b-a8c0-5aa2c68c0cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.413410 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c59852-930d-445b-a8c0-5aa2c68c0cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.414066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c59852-930d-445b-a8c0-5aa2c68c0cc8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:25.414361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c59852-930d-445b-a8c0-5aa2c68c0cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:25.415843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c59852-930d-445b-a8c0-5aa2c68c0cc8 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:25.425323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 339/678] 10.222.0.58 () {56 vars in 1115 bytes} [Fri Oct 28 18:54:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:25.430762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c59852-930d-445b-a8c0-5aa2c68c0cc8 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:25.455826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:25.472324 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 340/679] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:25.476306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648d822-a34e-4e2d-8549-ef62ab8e9b15 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.476592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648d822-a34e-4e2d-8549-ef62ab8e9b15 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.476866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648d822-a34e-4e2d-8549-ef62ab8e9b15 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.478362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 341/680] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:25.484010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12465f3b-c832-4a37-b806-c9c662f5d980 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.484317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12465f3b-c832-4a37-b806-c9c662f5d980 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.484563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12465f3b-c832-4a37-b806-c9c662f5d980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.505755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-12465f3b-c832-4a37-b806-c9c662f5d980 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:25.506996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-12465f3b-c832-4a37-b806-c9c662f5d980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:25.543803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c59852-930d-445b-a8c0-5aa2c68c0cc8 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:25.577729 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a59b94-b5a4-4a87-88ed-292f5a8f6f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:25.578650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a59b94-b5a4-4a87-88ed-292f5a8f6f49 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.579165 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a59b94-b5a4-4a87-88ed-292f5a8f6f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.579775 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a59b94-b5a4-4a87-88ed-292f5a8f6f49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.587205 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 342/681] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:25.591736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a59b94-b5a4-4a87-88ed-292f5a8f6f49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:25.593395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a59b94-b5a4-4a87-88ed-292f5a8f6f49 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-c6a59b94-b5a4-4a87-88ed-292f5a8f6f49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:25.608451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:25.609378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:25.610255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:25.610255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:25.610255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:25.612273 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 340/682] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:25.618078 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12465f3b-c832-4a37-b806-c9c662f5d980 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:25.665382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:25.725362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12465f3b-c832-4a37-b806-c9c662f5d980 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:25.746460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a59b94-b5a4-4a87-88ed-292f5a8f6f49 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:25.762437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5260925-d8a1-4cfe-9de6-6d829f2c44cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:25.763200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5260925-d8a1-4cfe-9de6-6d829f2c44cb None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.763480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5260925-d8a1-4cfe-9de6-6d829f2c44cb None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.763747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5260925-d8a1-4cfe-9de6-6d829f2c44cb None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.764387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5260925-d8a1-4cfe-9de6-6d829f2c44cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:25.764687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5260925-d8a1-4cfe-9de6-6d829f2c44cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:25.766205 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5260925-d8a1-4cfe-9de6-6d829f2c44cb None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:25.779869 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 343/683] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:25.785163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5260925-d8a1-4cfe-9de6-6d829f2c44cb None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:25.831884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:25.884941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a59b94-b5a4-4a87-88ed-292f5a8f6f49 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:25.914894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5260925-d8a1-4cfe-9de6-6d829f2c44cb None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:25.918336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:25.927874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345745a1-5a8c-4eed-afa5-f97150870330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:25.928805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345745a1-5a8c-4eed-afa5-f97150870330 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.929265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345745a1-5a8c-4eed-afa5-f97150870330 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.929809 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345745a1-5a8c-4eed-afa5-f97150870330 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.930928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345745a1-5a8c-4eed-afa5-f97150870330 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:25.932952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345745a1-5a8c-4eed-afa5-f97150870330 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-345745a1-5a8c-4eed-afa5-f97150870330 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:25.942442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:25.943641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:25.944474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 341/684] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:25.945266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345745a1-5a8c-4eed-afa5-f97150870330 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:25.954417 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a71f-1b80-4533-b0e2-546e76b54168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:25.955074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a71f-1b80-4533-b0e2-546e76b54168 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:25.955351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a71f-1b80-4533-b0e2-546e76b54168 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:25.955618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a71f-1b80-4533-b0e2-546e76b54168 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:25.962763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28a71f-1b80-4533-b0e2-546e76b54168 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:25.964228 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28a71f-1b80-4533-b0e2-546e76b54168 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-6e28a71f-1b80-4533-b0e2-546e76b54168 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:25.977203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:25.978169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 18:54:25.979167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:25.983716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 344/685] 10.222.0.58 () {56 vars in 1117 bytes} [Fri Oct 28 18:54:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:25.984321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a71f-1b80-4533-b0e2-546e76b54168 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.056686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345745a1-5a8c-4eed-afa5-f97150870330 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.083932 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:26.089868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a71f-1b80-4533-b0e2-546e76b54168 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.094742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3647626-0a4b-4c2b-b89e-f11bcc9811c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.095403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3647626-0a4b-4c2b-b89e-f11bcc9811c9 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.095680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3647626-0a4b-4c2b-b89e-f11bcc9811c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.095947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3647626-0a4b-4c2b-b89e-f11bcc9811c9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.096524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3647626-0a4b-4c2b-b89e-f11bcc9811c9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.098314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3647626-0a4b-4c2b-b89e-f11bcc9811c9 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-e3647626-0a4b-4c2b-b89e-f11bcc9811c9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:26.109386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:26.110336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:26.111675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 342/686] 10.222.0.58 () {56 vars in 1132 bytes} [Fri Oct 28 18:54:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:26.112679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3647626-0a4b-4c2b-b89e-f11bcc9811c9 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.130425 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d98afd-6466-43af-a7e4-b88820b2c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.131206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d98afd-6466-43af-a7e4-b88820b2c780 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.131483 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d98afd-6466-43af-a7e4-b88820b2c780 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.131750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d98afd-6466-43af-a7e4-b88820b2c780 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.132376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d98afd-6466-43af-a7e4-b88820b2c780 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:26.132699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d98afd-6466-43af-a7e4-b88820b2c780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.134038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d98afd-6466-43af-a7e4-b88820b2c780 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.140262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 345/687] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:26.141604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d98afd-6466-43af-a7e4-b88820b2c780 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.174907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:26.222267 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3647626-0a4b-4c2b-b89e-f11bcc9811c9 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.257230 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d98afd-6466-43af-a7e4-b88820b2c780 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.257456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f175ea0-82a9-4fa9-bce7-8dfb18d895fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.257921 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f175ea0-82a9-4fa9-bce7-8dfb18d895fb None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.258200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f175ea0-82a9-4fa9-bce7-8dfb18d895fb None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.258468 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f175ea0-82a9-4fa9-bce7-8dfb18d895fb None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.259221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f175ea0-82a9-4fa9-bce7-8dfb18d895fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:26.259525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f175ea0-82a9-4fa9-bce7-8dfb18d895fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.260966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f175ea0-82a9-4fa9-bce7-8dfb18d895fb None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.279685 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 343/688] 10.222.0.58 () {56 vars in 1149 bytes} [Fri Oct 28 18:54:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:26.280501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f175ea0-82a9-4fa9-bce7-8dfb18d895fb None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.298759 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745104c-7ccc-4e93-acfa-d3e1d8a29cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.299397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745104c-7ccc-4e93-acfa-d3e1d8a29cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.299672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745104c-7ccc-4e93-acfa-d3e1d8a29cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.299937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745104c-7ccc-4e93-acfa-d3e1d8a29cb0 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.300551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745104c-7ccc-4e93-acfa-d3e1d8a29cb0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:26.300847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745104c-7ccc-4e93-acfa-d3e1d8a29cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.302185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745104c-7ccc-4e93-acfa-d3e1d8a29cb0 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.306701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 346/689] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:26.307834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4053c9-1c69-4555-acb4-b899323cb53f None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.308648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4053c9-1c69-4555-acb4-b899323cb53f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.308926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4053c9-1c69-4555-acb4-b899323cb53f None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.310220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 347/690] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:26.310758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4053c9-1c69-4555-acb4-b899323cb53f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.333770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:26.383683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f175ea0-82a9-4fa9-bce7-8dfb18d895fb None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.401875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:26.411709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4053c9-1c69-4555-acb4-b899323cb53f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.416657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e6edb-01af-4b02-8cb6-ba7da538fa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.417251 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e6edb-01af-4b02-8cb6-ba7da538fa7c None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.418113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e6edb-01af-4b02-8cb6-ba7da538fa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.418391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e6edb-01af-4b02-8cb6-ba7da538fa7c None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.419035 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e6edb-01af-4b02-8cb6-ba7da538fa7c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:26.419334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e6edb-01af-4b02-8cb6-ba7da538fa7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.420676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e6edb-01af-4b02-8cb6-ba7da538fa7c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.425380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 344/691] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:26.426228 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e6edb-01af-4b02-8cb6-ba7da538fa7c None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.446423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d71d3-84ca-4e70-92f5-620126befc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.447041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d71d3-84ca-4e70-92f5-620126befc39 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.447322 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d71d3-84ca-4e70-92f5-620126befc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.447588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d71d3-84ca-4e70-92f5-620126befc39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.512186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d71d3-84ca-4e70-92f5-620126befc39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.513604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d71d3-84ca-4e70-92f5-620126befc39 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.522776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e6edb-01af-4b02-8cb6-ba7da538fa7c None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.535418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:26.556312 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f8269-0400-401e-90dd-e0b4a72d4a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.556908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f8269-0400-401e-90dd-e0b4a72d4a38 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.557190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f8269-0400-401e-90dd-e0b4a72d4a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.557456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f8269-0400-401e-90dd-e0b4a72d4a38 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.558073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f8269-0400-401e-90dd-e0b4a72d4a38 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:26.558955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f8269-0400-401e-90dd-e0b4a72d4a38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.560332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f8269-0400-401e-90dd-e0b4a72d4a38 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.564922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 345/692] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:26.566108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbcc19-43f0-462c-8b90-436c8b1e2288 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.566393 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbcc19-43f0-462c-8b90-436c8b1e2288 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.566661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbcc19-43f0-462c-8b90-436c8b1e2288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.583663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-17fbcc19-43f0-462c-8b90-436c8b1e2288 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:26.584832 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-17fbcc19-43f0-462c-8b90-436c8b1e2288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:26.603937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:26.614676 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 348/693] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:54:26 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:54:26.615508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d71d3-84ca-4e70-92f5-620126befc39 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.652719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 346/694] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:26.653455 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbcc19-43f0-462c-8b90-436c8b1e2288 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.717399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d71d3-84ca-4e70-92f5-620126befc39 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.759090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbcc19-43f0-462c-8b90-436c8b1e2288 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.762473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a1bba-6acd-405d-8ab6-dcac9f668b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.763116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a1bba-6acd-405d-8ab6-dcac9f668b1d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.763394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a1bba-6acd-405d-8ab6-dcac9f668b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.763661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a1bba-6acd-405d-8ab6-dcac9f668b1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.764350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a1bba-6acd-405d-8ab6-dcac9f668b1d None admin] RBAC: Adding query filter params (interface=public, service_id=936878a24d6a4872b96ded775b8217f0, region_id=RegionOne) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:26.764656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a1bba-6acd-405d-8ab6-dcac9f668b1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.766008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a1bba-6acd-405d-8ab6-dcac9f668b1d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.767277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:26.779093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 349/695] 10.222.0.58 () {56 vars in 1251 bytes} [Fri Oct 28 18:54:26 2022] GET /identity/v3/endpoints?service_id=936878a24d6a4872b96ded775b8217f0&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:26.780336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a1bba-6acd-405d-8ab6-dcac9f668b1d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.799535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6879d-6f7e-4dd2-a25d-e6da15914b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.800447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6879d-6f7e-4dd2-a25d-e6da15914b95 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.800726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6879d-6f7e-4dd2-a25d-e6da15914b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.801055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6879d-6f7e-4dd2-a25d-e6da15914b95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.801718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6879d-6f7e-4dd2-a25d-e6da15914b95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.803575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6879d-6f7e-4dd2-a25d-e6da15914b95 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.826307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 347/696] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 18:54:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:26.827305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6879d-6f7e-4dd2-a25d-e6da15914b95 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:26.853409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:26.916107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a1bba-6acd-405d-8ab6-dcac9f668b1d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.958601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6879d-6f7e-4dd2-a25d-e6da15914b95 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:26.961167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d20cd1-4a3d-41de-8780-01fa3d20b392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:26.961804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d20cd1-4a3d-41de-8780-01fa3d20b392 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:26.962074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d20cd1-4a3d-41de-8780-01fa3d20b392 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:26.962333 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d20cd1-4a3d-41de-8780-01fa3d20b392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:26.974720 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d20cd1-4a3d-41de-8780-01fa3d20b392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:26.976292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d20cd1-4a3d-41de-8780-01fa3d20b392 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-79d20cd1-4a3d-41de-8780-01fa3d20b392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:26.992146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:26.993156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:26.994061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:26.994061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:26.994061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:26.994061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 350/697] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:26.994061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d20cd1-4a3d-41de-8780-01fa3d20b392 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:27.006407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9c441-cd3f-4342-82fa-574658bdfa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.007032 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9c441-cd3f-4342-82fa-574658bdfa8d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.007313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9c441-cd3f-4342-82fa-574658bdfa8d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.007582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9c441-cd3f-4342-82fa-574658bdfa8d None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.008189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9c441-cd3f-4342-82fa-574658bdfa8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.009570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9c441-cd3f-4342-82fa-574658bdfa8d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.016419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 348/698] 10.222.0.58 () {56 vars in 1087 bytes} [Fri Oct 28 18:54:26 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 18:54:27.017222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9c441-cd3f-4342-82fa-574658bdfa8d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:27.051762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:27.138529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d20cd1-4a3d-41de-8780-01fa3d20b392 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:27.142264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:27.171514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9c441-cd3f-4342-82fa-574658bdfa8d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:27.192134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd4c96-a4b4-4ab5-8ba6-83f519a5c9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.192790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd4c96-a4b4-4ab5-8ba6-83f519a5c9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.193066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd4c96-a4b4-4ab5-8ba6-83f519a5c9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.193339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd4c96-a4b4-4ab5-8ba6-83f519a5c9b0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.195083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd4c96-a4b4-4ab5-8ba6-83f519a5c9b0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.195821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd4c96-a4b4-4ab5-8ba6-83f519a5c9b0 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-cdfd4c96-a4b4-4ab5-8ba6-83f519a5c9b0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:27.206536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:27.207576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:27.208402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 351/699] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:54:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:27.212090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9b4ac-542d-4a19-a704-2fea6ea26347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.212702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9b4ac-542d-4a19-a704-2fea6ea26347 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.212981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9b4ac-542d-4a19-a704-2fea6ea26347 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.213251 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9b4ac-542d-4a19-a704-2fea6ea26347 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.213893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9b4ac-542d-4a19-a704-2fea6ea26347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:27.214194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9b4ac-542d-4a19-a704-2fea6ea26347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.216166 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9b4ac-542d-4a19-a704-2fea6ea26347 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.218868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd4c96-a4b4-4ab5-8ba6-83f519a5c9b0 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:27.232341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 349/700] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:27.237893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9b4ac-542d-4a19-a704-2fea6ea26347 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:27.332855 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:27.336351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd4c96-a4b4-4ab5-8ba6-83f519a5c9b0 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:27.352915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9b4ac-542d-4a19-a704-2fea6ea26347 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:27.379678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c810d7-127a-4b3a-bc5b-3aea32d6fe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.380386 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c810d7-127a-4b3a-bc5b-3aea32d6fe7f None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.380736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c810d7-127a-4b3a-bc5b-3aea32d6fe7f None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.381083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c810d7-127a-4b3a-bc5b-3aea32d6fe7f None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.381787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c810d7-127a-4b3a-bc5b-3aea32d6fe7f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:27.382157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c810d7-127a-4b3a-bc5b-3aea32d6fe7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.383846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c810d7-127a-4b3a-bc5b-3aea32d6fe7f None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.390267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 352/701] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:54:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:27.396408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c810d7-127a-4b3a-bc5b-3aea32d6fe7f None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:27.398896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479e95e-ba99-4247-8754-c7dee74329d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.399716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479e95e-ba99-4247-8754-c7dee74329d5 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.400184 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479e95e-ba99-4247-8754-c7dee74329d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.400458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479e95e-ba99-4247-8754-c7dee74329d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.408069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479e95e-ba99-4247-8754-c7dee74329d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.409693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479e95e-ba99-4247-8754-c7dee74329d5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-f479e95e-ba99-4247-8754-c7dee74329d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:27.421780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:27.422695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:27.423601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:27.423601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:27.423601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:27.423601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:27.423601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:27.423601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:27.423601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:27.423601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:27.423601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 350/702] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:27.431893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479e95e-ba99-4247-8754-c7dee74329d5 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:27.437037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:27.519809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c810d7-127a-4b3a-bc5b-3aea32d6fe7f None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:27.537158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479e95e-ba99-4247-8754-c7dee74329d5 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:27.560716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3aa84-7f03-4d5f-8f70-6375fb69c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.561664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3aa84-7f03-4d5f-8f70-6375fb69c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.562158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3aa84-7f03-4d5f-8f70-6375fb69c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.562675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3aa84-7f03-4d5f-8f70-6375fb69c8f1 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.563531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3aa84-7f03-4d5f-8f70-6375fb69c8f1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:27.564119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3aa84-7f03-4d5f-8f70-6375fb69c8f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.566228 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3aa84-7f03-4d5f-8f70-6375fb69c8f1 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.577372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda05f1-7cbf-458b-a818-5a243cd3424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.577372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda05f1-7cbf-458b-a818-5a243cd3424f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.577372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda05f1-7cbf-458b-a818-5a243cd3424f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.577372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda05f1-7cbf-458b-a818-5a243cd3424f None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.577791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda05f1-7cbf-458b-a818-5a243cd3424f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:27.577791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda05f1-7cbf-458b-a818-5a243cd3424f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.577791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda05f1-7cbf-458b-a818-5a243cd3424f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.578114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 353/703] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:54:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:27.579132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3aa84-7f03-4d5f-8f70-6375fb69c8f1 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:27.596354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 351/704] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:27.597374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda05f1-7cbf-458b-a818-5a243cd3424f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:27.607316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:27.695511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3aa84-7f03-4d5f-8f70-6375fb69c8f1 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:27.712108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda05f1-7cbf-458b-a818-5a243cd3424f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:27.723332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:27.741218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d86fa3-213b-422c-a4e1-4ff8da44a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.742461 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d86fa3-213b-422c-a4e1-4ff8da44a9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.742963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d86fa3-213b-422c-a4e1-4ff8da44a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.743479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d86fa3-213b-422c-a4e1-4ff8da44a9b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.744390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d86fa3-213b-422c-a4e1-4ff8da44a9b6 None admin] RBAC: Adding query filter params (interface=public, service_id=31f81dda3a2b46a3994bfa9ec5b900b4, region_id=RegionOne) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:27.744794 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d86fa3-213b-422c-a4e1-4ff8da44a9b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.746277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d86fa3-213b-422c-a4e1-4ff8da44a9b6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.759881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c82c5-41bc-4511-84f8-d099cdfcac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.760481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 354/705] 10.222.0.58 () {56 vars in 1251 bytes} [Fri Oct 28 18:54:27 2022] GET /identity/v3/endpoints?service_id=31f81dda3a2b46a3994bfa9ec5b900b4&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:27.761298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c82c5-41bc-4511-84f8-d099cdfcac8c None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.761422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d86fa3-213b-422c-a4e1-4ff8da44a9b6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:27.761960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c82c5-41bc-4511-84f8-d099cdfcac8c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.762437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c82c5-41bc-4511-84f8-d099cdfcac8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.763401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c82c5-41bc-4511-84f8-d099cdfcac8c None admin] RBAC: Adding query filter params (role.id=2604cfbe6860499e9e894574ea9c0665, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=0eef740ccf884c079e6a2d6cdc89f4bb) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:27.764108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c82c5-41bc-4511-84f8-d099cdfcac8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.766645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c82c5-41bc-4511-84f8-d099cdfcac8c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.775257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 352/706] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:27 2022] GET /identity/v3/role_assignments?user.id=0eef740ccf884c079e6a2d6cdc89f4bb&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=2604cfbe6860499e9e894574ea9c0665 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:27.881793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d86fa3-213b-422c-a4e1-4ff8da44a9b6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:27.919653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fc672-8dcb-4cfc-9b79-b1aa219b523b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:27.920265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fc672-8dcb-4cfc-9b79-b1aa219b523b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:27.920541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fc672-8dcb-4cfc-9b79-b1aa219b523b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:27.920809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fc672-8dcb-4cfc-9b79-b1aa219b523b None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:27.921423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047fc672-8dcb-4cfc-9b79-b1aa219b523b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:27.922831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047fc672-8dcb-4cfc-9b79-b1aa219b523b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:27.929865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 355/707] 10.222.0.58 () {56 vars in 1087 bytes} [Fri Oct 28 18:54:27 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 18:54:29.137395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c84f86-96b8-4715-b5b5-1ec2f7588b45 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:29.138420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c84f86-96b8-4715-b5b5-1ec2f7588b45 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:29.138906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c84f86-96b8-4715-b5b5-1ec2f7588b45 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:29.140910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 353/708] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:29.147315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69183a0a-2c2b-448f-890f-1f77e88eefe6 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:29.148316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69183a0a-2c2b-448f-890f-1f77e88eefe6 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:29.148789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69183a0a-2c2b-448f-890f-1f77e88eefe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:29.169156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-69183a0a-2c2b-448f-890f-1f77e88eefe6 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:29.170903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-69183a0a-2c2b-448f-890f-1f77e88eefe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:29.195336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:29.247894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 356/709] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:29.270239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d3009-924a-4faf-bab8-8a9230698630 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:29.270520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d3009-924a-4faf-bab8-8a9230698630 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:29.270787 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d3009-924a-4faf-bab8-8a9230698630 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:29.272430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 354/710] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:29.277650 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a197001-1087-43a1-9338-961f8a887f10 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:29.277932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a197001-1087-43a1-9338-961f8a887f10 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:29.278202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a197001-1087-43a1-9338-961f8a887f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:29.296402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-5a197001-1087-43a1-9338-961f8a887f10 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:29.297564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5a197001-1087-43a1-9338-961f8a887f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:29.364133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 357/711] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:29.520989 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d3009-924a-4faf-bab8-8a9230698630 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:29.588804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:29.626470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d3009-924a-4faf-bab8-8a9230698630 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:29.641715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a197001-1087-43a1-9338-961f8a887f10 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:29.661938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac69ad-71ff-46fc-a0ea-113c512f5b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:29.662562 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac69ad-71ff-46fc-a0ea-113c512f5b2b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:29.662836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac69ad-71ff-46fc-a0ea-113c512f5b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:29.663131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac69ad-71ff-46fc-a0ea-113c512f5b2b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:29.663708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac69ad-71ff-46fc-a0ea-113c512f5b2b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:29.665182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac69ad-71ff-46fc-a0ea-113c512f5b2b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-97ac69ad-71ff-46fc-a0ea-113c512f5b2b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:29.675404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:29.676390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:29.677806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 355/712] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:29.684560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac69ad-71ff-46fc-a0ea-113c512f5b2b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:29.750897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:29.772075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a197001-1087-43a1-9338-961f8a887f10 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:29.811099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac69ad-71ff-46fc-a0ea-113c512f5b2b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:29.813936 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c85e19-a39b-4989-a4f7-db3ff735f304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:29.814647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c85e19-a39b-4989-a4f7-db3ff735f304 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:29.815077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c85e19-a39b-4989-a4f7-db3ff735f304 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:29.815417 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c85e19-a39b-4989-a4f7-db3ff735f304 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:29.816074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c85e19-a39b-4989-a4f7-db3ff735f304 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:29.817525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c85e19-a39b-4989-a4f7-db3ff735f304 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-e9c85e19-a39b-4989-a4f7-db3ff735f304 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:29.828602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:29.829746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:29.830596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 358/713] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:29.837761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c85e19-a39b-4989-a4f7-db3ff735f304 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:29.849848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:29.858109 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbef36c-3c6f-45fd-8da9-d23850d1075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:29.858720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbef36c-3c6f-45fd-8da9-d23850d1075a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:29.859032 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbef36c-3c6f-45fd-8da9-d23850d1075a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:29.859311 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbef36c-3c6f-45fd-8da9-d23850d1075a None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:29.859945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbef36c-3c6f-45fd-8da9-d23850d1075a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:29.860250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbef36c-3c6f-45fd-8da9-d23850d1075a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:29.861618 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbef36c-3c6f-45fd-8da9-d23850d1075a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:29.866997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 356/714] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:29.873130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbef36c-3c6f-45fd-8da9-d23850d1075a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:29.956615 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c85e19-a39b-4989-a4f7-db3ff735f304 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:29.987138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbef36c-3c6f-45fd-8da9-d23850d1075a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:29.993263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f374ca-b8cc-43c6-b8ad-503ae22deb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:29.994121 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f374ca-b8cc-43c6-b8ad-503ae22deb8f None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:29.994613 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f374ca-b8cc-43c6-b8ad-503ae22deb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:29.995132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f374ca-b8cc-43c6-b8ad-503ae22deb8f None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:29.996026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f374ca-b8cc-43c6-b8ad-503ae22deb8f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:29.996545 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f374ca-b8cc-43c6-b8ad-503ae22deb8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:29.998542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f374ca-b8cc-43c6-b8ad-503ae22deb8f None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:30.008548 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 359/715] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:30.010041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2736a5-a892-4910-a256-bbf1864ea8f5 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.010523 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2736a5-a892-4910-a256-bbf1864ea8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.011039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2736a5-a892-4910-a256-bbf1864ea8f5 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.012489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 360/716] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:30.013365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2736a5-a892-4910-a256-bbf1864ea8f5 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:30.028925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcdcad-95b1-4d7b-81eb-d71629a42cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:30.029220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:30.029938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcdcad-95b1-4d7b-81eb-d71629a42cfd None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.030326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcdcad-95b1-4d7b-81eb-d71629a42cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.030546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcdcad-95b1-4d7b-81eb-d71629a42cfd None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.032052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcdcad-95b1-4d7b-81eb-d71629a42cfd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:30.032413 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcdcad-95b1-4d7b-81eb-d71629a42cfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:30.034149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcdcad-95b1-4d7b-81eb-d71629a42cfd None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:30.046508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 357/717] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:30.046508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7af82-0279-4b43-ad63-78f3046033cd None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.046508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7af82-0279-4b43-ad63-78f3046033cd None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.046508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7af82-0279-4b43-ad63-78f3046033cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.072645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d3d7af82-0279-4b43-ad63-78f3046033cd None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:30.074235 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d3d7af82-0279-4b43-ad63-78f3046033cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:30.139327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2736a5-a892-4910-a256-bbf1864ea8f5 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:30.139594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:30.167596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 358/718] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:30.178143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8e7d2-cdcd-4077-8a97-7046b32edb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:30.178849 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8e7d2-cdcd-4077-8a97-7046b32edb47 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.179177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8e7d2-cdcd-4077-8a97-7046b32edb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.179446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8e7d2-cdcd-4077-8a97-7046b32edb47 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.180094 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8e7d2-cdcd-4077-8a97-7046b32edb47 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:30.180444 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8e7d2-cdcd-4077-8a97-7046b32edb47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:30.181836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8e7d2-cdcd-4077-8a97-7046b32edb47 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:30.188248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 361/719] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:30.193657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7af82-0279-4b43-ad63-78f3046033cd None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:30.194951 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db3a86-576c-4eab-a50a-48ada0c49864 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.195281 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db3a86-576c-4eab-a50a-48ada0c49864 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.195551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db3a86-576c-4eab-a50a-48ada0c49864 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.197163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 362/720] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:30.204488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6bd4-0fd8-4c41-befe-c5e9f5da972e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.204816 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6bd4-0fd8-4c41-befe-c5e9f5da972e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.205026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6bd4-0fd8-4c41-befe-c5e9f5da972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.228696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-7a7f6bd4-0fd8-4c41-befe-c5e9f5da972e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:30.229523 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7a7f6bd4-0fd8-4c41-befe-c5e9f5da972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:30.305927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 363/721] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:30.312287 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7af82-0279-4b43-ad63-78f3046033cd None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:30.348159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af404d9-94b8-4b7f-b834-f72cecfc3651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:30.348760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af404d9-94b8-4b7f-b834-f72cecfc3651 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.349036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af404d9-94b8-4b7f-b834-f72cecfc3651 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.349302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af404d9-94b8-4b7f-b834-f72cecfc3651 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.349970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af404d9-94b8-4b7f-b834-f72cecfc3651 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:30.351442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af404d9-94b8-4b7f-b834-f72cecfc3651 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:30.377862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 359/722] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 18:54:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:30.445804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f6bd4-0fd8-4c41-befe-c5e9f5da972e None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:30.455648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:30.561412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f6bd4-0fd8-4c41-befe-c5e9f5da972e None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:30.602816 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6effc-7fee-4c15-94b8-e94241370883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:30.603538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6effc-7fee-4c15-94b8-e94241370883 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.603903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6effc-7fee-4c15-94b8-e94241370883 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.604173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6effc-7fee-4c15-94b8-e94241370883 None admin] PATH_INFO: `/v3/services/image` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.604754 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6effc-7fee-4c15-94b8-e94241370883 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:30.605055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af404d9-94b8-4b7f-b834-f72cecfc3651 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:30.606225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6effc-7fee-4c15-94b8-e94241370883 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-89c6effc-7fee-4c15-94b8-e94241370883 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:30.616084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:30.617091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:30.617915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 364/723] 10.222.0.58 () {56 vars in 1105 bytes} [Fri Oct 28 18:54:30 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:30.623785 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6effc-7fee-4c15-94b8-e94241370883 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:30.659579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:30.703249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:30.735389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af404d9-94b8-4b7f-b834-f72cecfc3651 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:30.743881 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6effc-7fee-4c15-94b8-e94241370883 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:30.786210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d12d3-cb2f-4966-b176-895414c073d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:30.787123 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d12d3-cb2f-4966-b176-895414c073d7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.787550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d12d3-cb2f-4966-b176-895414c073d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.787967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d12d3-cb2f-4966-b176-895414c073d7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.788173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac54dc1-8ea7-4378-bb3d-96fb68f59350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:30.788777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac54dc1-8ea7-4378-bb3d-96fb68f59350 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.789055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac54dc1-8ea7-4378-bb3d-96fb68f59350 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.789174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d12d3-cb2f-4966-b176-895414c073d7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:30.789317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac54dc1-8ea7-4378-bb3d-96fb68f59350 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.789940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac54dc1-8ea7-4378-bb3d-96fb68f59350 None admin] RBAC: Adding query filter params (name=image) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:30.790235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac54dc1-8ea7-4378-bb3d-96fb68f59350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:30.791133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d12d3-cb2f-4966-b176-895414c073d7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:30.791601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac54dc1-8ea7-4378-bb3d-96fb68f59350 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:30.802498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 365/724] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:30.803617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac54dc1-8ea7-4378-bb3d-96fb68f59350 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-074d12d3-cb2f-4966-b176-895414c073d7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:30.805888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:30.806830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 18:54:30.807711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:30.811221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 360/725] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:54:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:30.819447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d12d3-cb2f-4966-b176-895414c073d7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:30.923701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac54dc1-8ea7-4378-bb3d-96fb68f59350 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:30.944894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d12d3-cb2f-4966-b176-895414c073d7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:30.956034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:30.956590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:30.964791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc12c-f8c5-4df0-9f63-1400da035fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:30.965713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc12c-f8c5-4df0-9f63-1400da035fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.966208 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc12c-f8c5-4df0-9f63-1400da035fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.966731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc12c-f8c5-4df0-9f63-1400da035fa6 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.967739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfc12c-f8c5-4df0-9f63-1400da035fa6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:30.968243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfc12c-f8c5-4df0-9f63-1400da035fa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:30.970852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfc12c-f8c5-4df0-9f63-1400da035fa6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:30.983408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 366/726] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:30.984928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a34f5d-43e8-4046-8944-ce9d07773dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:30.985558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a34f5d-43e8-4046-8944-ce9d07773dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:30.985834 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a34f5d-43e8-4046-8944-ce9d07773dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:30.986104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a34f5d-43e8-4046-8944-ce9d07773dd8 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:30.986749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a34f5d-43e8-4046-8944-ce9d07773dd8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:30.987095 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a34f5d-43e8-4046-8944-ce9d07773dd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:30.988519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a34f5d-43e8-4046-8944-ce9d07773dd8 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:30.994558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 361/727] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:54:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:31.000981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc12c-f8c5-4df0-9f63-1400da035fa6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:31.108199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc12c-f8c5-4df0-9f63-1400da035fa6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:31.155301 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe254f7-70ba-4b95-a094-c641be7282c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:31.162644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe254f7-70ba-4b95-a094-c641be7282c9 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:31.162644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe254f7-70ba-4b95-a094-c641be7282c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:31.162644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe254f7-70ba-4b95-a094-c641be7282c9 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:31.162644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe254f7-70ba-4b95-a094-c641be7282c9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:31.162644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe254f7-70ba-4b95-a094-c641be7282c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:31.162644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe254f7-70ba-4b95-a094-c641be7282c9 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:31.168526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 367/728] 10.222.0.58 () {56 vars in 1114 bytes} [Fri Oct 28 18:54:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:31.176517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a34f5d-43e8-4046-8944-ce9d07773dd8 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:31.393906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a34f5d-43e8-4046-8944-ce9d07773dd8 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:31.436652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddef8e-5690-4d98-b16d-8e831873e299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:31.443001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddef8e-5690-4d98-b16d-8e831873e299 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:31.443001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddef8e-5690-4d98-b16d-8e831873e299 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:31.443001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddef8e-5690-4d98-b16d-8e831873e299 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:31.443001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddef8e-5690-4d98-b16d-8e831873e299 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:31.443001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddef8e-5690-4d98-b16d-8e831873e299 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:31.488825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 362/729] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 18:54:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:32.384438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780883df-e4d5-4921-b3eb-502e3feb8146 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:32.385289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780883df-e4d5-4921-b3eb-502e3feb8146 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:32.385708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780883df-e4d5-4921-b3eb-502e3feb8146 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:32.388079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 368/730] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:32.394154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034b397-1cee-4b6c-bdb0-c8341793eaa3 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:32.395021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034b397-1cee-4b6c-bdb0-c8341793eaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:32.395513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034b397-1cee-4b6c-bdb0-c8341793eaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:32.414736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2034b397-1cee-4b6c-bdb0-c8341793eaa3 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:32.416347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2034b397-1cee-4b6c-bdb0-c8341793eaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:32.441875 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:32.499080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 363/731] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:32.721790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866b13f-fade-4852-9b8b-3de5e050ece7 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:32.722412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866b13f-fade-4852-9b8b-3de5e050ece7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:32.722894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866b13f-fade-4852-9b8b-3de5e050ece7 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:32.724781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 369/732] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:32.740758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49eccc-02ad-490f-addd-3e5dabeb508a None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:32.741330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49eccc-02ad-490f-addd-3e5dabeb508a None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:32.741829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49eccc-02ad-490f-addd-3e5dabeb508a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:32.767523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-bc49eccc-02ad-490f-addd-3e5dabeb508a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:32.768949 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-bc49eccc-02ad-490f-addd-3e5dabeb508a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:32.770558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866b13f-fade-4852-9b8b-3de5e050ece7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:32.835391 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:32.840586 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 364/733] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:32.875897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866b13f-fade-4852-9b8b-3de5e050ece7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:32.910157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfde8b7-2aee-4a13-b7cb-5d316ec2d858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:32.910761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfde8b7-2aee-4a13-b7cb-5d316ec2d858 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:32.911057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfde8b7-2aee-4a13-b7cb-5d316ec2d858 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:32.911326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfde8b7-2aee-4a13-b7cb-5d316ec2d858 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:32.911975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfde8b7-2aee-4a13-b7cb-5d316ec2d858 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:32.913329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfde8b7-2aee-4a13-b7cb-5d316ec2d858 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:32.929120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 370/734] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:54:32 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:54:33.122190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49eccc-02ad-490f-addd-3e5dabeb508a None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:33.214753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:33.232202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49eccc-02ad-490f-addd-3e5dabeb508a None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:33.273839 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c34089-558e-4a71-84b5-0cd8b6cb5de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:33.274472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c34089-558e-4a71-84b5-0cd8b6cb5de3 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.274747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c34089-558e-4a71-84b5-0cd8b6cb5de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.275036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c34089-558e-4a71-84b5-0cd8b6cb5de3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.275674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c34089-558e-4a71-84b5-0cd8b6cb5de3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:33.277606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c34089-558e-4a71-84b5-0cd8b6cb5de3 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-d0c34089-558e-4a71-84b5-0cd8b6cb5de3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:33.286765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:33.287795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:33.288651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:33.292934 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 365/735] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:33.294263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfde8b7-2aee-4a13-b7cb-5d316ec2d858 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:33.304167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fdf31-44ba-4313-b293-b30680bb93e7 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.304500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fdf31-44ba-4313-b293-b30680bb93e7 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.304779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fdf31-44ba-4313-b293-b30680bb93e7 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.306397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 366/736] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:54:33.311881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ba799-abcf-474e-8e1d-7fa0e8d8ae4c None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.312159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ba799-abcf-474e-8e1d-7fa0e8d8ae4c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.312425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ba799-abcf-474e-8e1d-7fa0e8d8ae4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.331502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f60ba799-abcf-474e-8e1d-7fa0e8d8ae4c None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:33.332671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f60ba799-abcf-474e-8e1d-7fa0e8d8ae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:33.405037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfde8b7-2aee-4a13-b7cb-5d316ec2d858 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:33.407483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 367/737] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:33.423754 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:33.440799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bff53b-f711-401d-8127-02f8ed3cf82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:33.441414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bff53b-f711-401d-8127-02f8ed3cf82e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.441692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bff53b-f711-401d-8127-02f8ed3cf82e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.441961 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bff53b-f711-401d-8127-02f8ed3cf82e None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.442593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bff53b-f711-401d-8127-02f8ed3cf82e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:33.442894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bff53b-f711-401d-8127-02f8ed3cf82e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:33.444299 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bff53b-f711-401d-8127-02f8ed3cf82e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:33.454552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 371/738] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:33.460150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ba799-abcf-474e-8e1d-7fa0e8d8ae4c None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:33.539378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:33.566067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ba799-abcf-474e-8e1d-7fa0e8d8ae4c None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:33.601407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35940ed-753c-4746-a08a-f6782a4ca34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:33.602371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35940ed-753c-4746-a08a-f6782a4ca34f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.602871 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35940ed-753c-4746-a08a-f6782a4ca34f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.603369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35940ed-753c-4746-a08a-f6782a4ca34f None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.604204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35940ed-753c-4746-a08a-f6782a4ca34f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:33.604746 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35940ed-753c-4746-a08a-f6782a4ca34f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:33.606934 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35940ed-753c-4746-a08a-f6782a4ca34f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:33.613346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 368/739] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:33.619876 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bff53b-f711-401d-8127-02f8ed3cf82e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:33.698579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35940ed-753c-4746-a08a-f6782a4ca34f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:33.726112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bff53b-f711-401d-8127-02f8ed3cf82e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:33.774359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c1050-35b9-4c7f-986b-0a441d871891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:33.775352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c1050-35b9-4c7f-986b-0a441d871891 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.775796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c1050-35b9-4c7f-986b-0a441d871891 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.776213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c1050-35b9-4c7f-986b-0a441d871891 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.777642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c1050-35b9-4c7f-986b-0a441d871891 None admin] RBAC: Adding query filter params (interface=admin, service_id=936878a24d6a4872b96ded775b8217f0, region_id=RegionOne) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:33.778099 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c1050-35b9-4c7f-986b-0a441d871891 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:33.779783 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c1050-35b9-4c7f-986b-0a441d871891 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:33.789738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 372/740] 10.222.0.58 () {56 vars in 1249 bytes} [Fri Oct 28 18:54:33 2022] GET /identity/v3/endpoints?service_id=936878a24d6a4872b96ded775b8217f0&interface=admin®ion_id=RegionOne => generated 193 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:33.789738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8a238-2e8a-40c4-9bcd-ff8844086b46 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.789738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8a238-2e8a-40c4-9bcd-ff8844086b46 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.789738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8a238-2e8a-40c4-9bcd-ff8844086b46 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.790450 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 373/741] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:33.791721 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8a238-2e8a-40c4-9bcd-ff8844086b46 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:33.819679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35940ed-753c-4746-a08a-f6782a4ca34f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:33.841938 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:33.856567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e0ba3-8ee5-460f-b2f1-c5511340ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:33.857313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e0ba3-8ee5-460f-b2f1-c5511340ce3a None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.857587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e0ba3-8ee5-460f-b2f1-c5511340ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.857857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e0ba3-8ee5-460f-b2f1-c5511340ce3a None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.858505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e0ba3-8ee5-460f-b2f1-c5511340ce3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:33.859884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e0ba3-8ee5-460f-b2f1-c5511340ce3a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:33.869063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:33.878054 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 369/742] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:54:33 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:54:33.880352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ae198-019f-47e2-92fa-6dd286b6764e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.880936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ae198-019f-47e2-92fa-6dd286b6764e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.881395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ae198-019f-47e2-92fa-6dd286b6764e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.903895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4a1ae198-019f-47e2-92fa-6dd286b6764e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:33.905525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4a1ae198-019f-47e2-92fa-6dd286b6764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:33.913824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8a238-2e8a-40c4-9bcd-ff8844086b46 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:33.957871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856c83d-18b3-46f3-9218-1d34400c1e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:33.958565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856c83d-18b3-46f3-9218-1d34400c1e54 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:33.958943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856c83d-18b3-46f3-9218-1d34400c1e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:33.959339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856c83d-18b3-46f3-9218-1d34400c1e54 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:33.960057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856c83d-18b3-46f3-9218-1d34400c1e54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:33.961658 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856c83d-18b3-46f3-9218-1d34400c1e54 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:33.977889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 374/743] 10.222.0.58 () {56 vars in 1087 bytes} [Fri Oct 28 18:54:33 2022] GET /identity/v3/services => generated 1957 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 18:54:33.996988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 370/744] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:33 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:34.269416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856c83d-18b3-46f3-9218-1d34400c1e54 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:34.375859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856c83d-18b3-46f3-9218-1d34400c1e54 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:34.383188 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:34.411160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:34.411842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:34.412084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:34.412359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:34.413009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:34.414421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:34.417055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING keystone.server.flask.common [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 28 18:54:34.436421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.sql.core [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 28 18:54:34.436421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 28 18:54:34.436421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [parameters: {'id': '3cc869d1ea7441b99afa238cb7f03f98', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 28 18:54:34.436421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3cc869d1ea7441b99afa238cb7f03f98.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3cc869d1ea7441b99afa238cb7f03f98. Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:54:34.444321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application result.read() Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 28 18:54:34.445112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application yield resource Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.session.rollback() Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.force_reraise() Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise self.value Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application session.commit() Oct 28 18:54:34.446046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._prepare_impl() Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.session.flush() Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._flush(objects) Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application compat.raise_( Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application flush_context.execute() Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 28 18:54:34.446943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rec.execute(self) Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 28 18:54:34.449158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application result.read() Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:54:34.450271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [parameters: {'id': '3cc869d1ea7441b99afa238cb7f03f98', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:34.452422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:34.454293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3cc869d1ea7441b99afa238cb7f03f98. Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:34.460092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 375/745] 10.222.0.58 () {60 vars in 1140 bytes} [Fri Oct 28 18:54:34 2022] POST /identity/v3/projects => generated 262 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:34.466140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ae198-019f-47e2-92fa-6dd286b6764e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:34.571301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ae198-019f-47e2-92fa-6dd286b6764e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:34.604458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:34.614013 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a806f8a-1523-495a-88bd-73be9a6930b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:34.614652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a806f8a-1523-495a-88bd-73be9a6930b4 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:34.614931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a806f8a-1523-495a-88bd-73be9a6930b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:34.615280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a806f8a-1523-495a-88bd-73be9a6930b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:34.622365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a806f8a-1523-495a-88bd-73be9a6930b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:34.623952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a806f8a-1523-495a-88bd-73be9a6930b4 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-6a806f8a-1523-495a-88bd-73be9a6930b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:34.635463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:34.636884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:34.637745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:34.637745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:34.637745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:34.637745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:34.637745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:34.637745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:34.637745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:34.637745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:34.637745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 371/746] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:34.644662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:34.763220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd39fd-35ad-410b-872d-180b21ee6074 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:34.799479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468b14f-747d-4340-a40d-f9daf1da46f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:34.800473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468b14f-747d-4340-a40d-f9daf1da46f2 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:34.801031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468b14f-747d-4340-a40d-f9daf1da46f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:34.801505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468b14f-747d-4340-a40d-f9daf1da46f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:34.802398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468b14f-747d-4340-a40d-f9daf1da46f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:34.802912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468b14f-747d-4340-a40d-f9daf1da46f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:34.804861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468b14f-747d-4340-a40d-f9daf1da46f2 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:34.828084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 376/747] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:34 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:35.230469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281f2fc-f487-4d91-9261-5c796c4881c0 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:35.230655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281f2fc-f487-4d91-9261-5c796c4881c0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:35.230936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281f2fc-f487-4d91-9261-5c796c4881c0 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:35.232542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 372/748] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:35.238038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a157868-d7d7-4662-938c-56e746916acc None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:35.238326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a157868-d7d7-4662-938c-56e746916acc None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:35.238597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a157868-d7d7-4662-938c-56e746916acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:35.257860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-3a157868-d7d7-4662-938c-56e746916acc None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:35.259690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3a157868-d7d7-4662-938c-56e746916acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:35.277904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:35.330661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 377/749] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:35 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:35.599985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281f2fc-f487-4d91-9261-5c796c4881c0 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:35.704044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281f2fc-f487-4d91-9261-5c796c4881c0 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:35.743171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c886b30-8baa-4a7e-9aa7-1c682de2fbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:35.743807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c886b30-8baa-4a7e-9aa7-1c682de2fbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:35.744091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c886b30-8baa-4a7e-9aa7-1c682de2fbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:35.744348 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c886b30-8baa-4a7e-9aa7-1c682de2fbb8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:35.744932 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c886b30-8baa-4a7e-9aa7-1c682de2fbb8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:35.746872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c886b30-8baa-4a7e-9aa7-1c682de2fbb8 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-1c886b30-8baa-4a7e-9aa7-1c682de2fbb8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:35.756142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:35.757072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:35.757893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 373/750] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:35.764008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a157868-d7d7-4662-938c-56e746916acc None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:35.875481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a157868-d7d7-4662-938c-56e746916acc None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:35.910220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c2266-cf8b-457b-9300-13a7d60a1d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:35.910829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c2266-cf8b-457b-9300-13a7d60a1d2b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:35.911135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c2266-cf8b-457b-9300-13a7d60a1d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:35.911405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c2266-cf8b-457b-9300-13a7d60a1d2b None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:35.912033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c2266-cf8b-457b-9300-13a7d60a1d2b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:35.912333 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c2266-cf8b-457b-9300-13a7d60a1d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:35.913703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c2266-cf8b-457b-9300-13a7d60a1d2b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:35.919887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 378/751] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:35.925820 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c886b30-8baa-4a7e-9aa7-1c682de2fbb8 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:35.981506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:36.033544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c886b30-8baa-4a7e-9aa7-1c682de2fbb8 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:36.068112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20774eb3-2ab5-45ec-b121-7068d050ce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:36.068783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20774eb3-2ab5-45ec-b121-7068d050ce0f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.069061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20774eb3-2ab5-45ec-b121-7068d050ce0f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.069331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20774eb3-2ab5-45ec-b121-7068d050ce0f None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.070125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20774eb3-2ab5-45ec-b121-7068d050ce0f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:36.070430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20774eb3-2ab5-45ec-b121-7068d050ce0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:36.071881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20774eb3-2ab5-45ec-b121-7068d050ce0f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:36.077167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 374/752] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:36.083108 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c2266-cf8b-457b-9300-13a7d60a1d2b None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:36.102755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:36.106754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae5792-56b9-472c-82da-af71e1918fc3 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.107074 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae5792-56b9-472c-82da-af71e1918fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.107352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae5792-56b9-472c-82da-af71e1918fc3 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.108939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 375/753] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:54:36.115107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6530f1-07de-4e66-a368-1551d0a529a0 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.115388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6530f1-07de-4e66-a368-1551d0a529a0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.115927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6530f1-07de-4e66-a368-1551d0a529a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.140445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-de6530f1-07de-4e66-a368-1551d0a529a0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:36.142115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-de6530f1-07de-4e66-a368-1551d0a529a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:36.196336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c2266-cf8b-457b-9300-13a7d60a1d2b None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:36.223500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 376/754] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:36 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:36.235887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28a105-41dc-4781-89b1-55be10c39df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:36.236941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a105-41dc-4781-89b1-55be10c39df9 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.237510 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a105-41dc-4781-89b1-55be10c39df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.238029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a105-41dc-4781-89b1-55be10c39df9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.238951 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28a105-41dc-4781-89b1-55be10c39df9 None admin] RBAC: Adding query filter params (interface=public, service_id=35edbbc5566b4b1f98188858a590f1c1, region_id=RegionOne) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:36.239508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28a105-41dc-4781-89b1-55be10c39df9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:36.241204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28a105-41dc-4781-89b1-55be10c39df9 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:36.248115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 379/755] 10.222.0.58 () {56 vars in 1251 bytes} [Fri Oct 28 18:54:36 2022] GET /identity/v3/endpoints?service_id=35edbbc5566b4b1f98188858a590f1c1&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:36.255040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6530f1-07de-4e66-a368-1551d0a529a0 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:36.302793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:36.348642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95c3ba-cd04-41e9-a5a8-443cc6d3fbba None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.348992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95c3ba-cd04-41e9-a5a8-443cc6d3fbba None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.349186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95c3ba-cd04-41e9-a5a8-443cc6d3fbba None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.351841 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 380/756] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:54:36.355821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c42bcd-2a34-43d1-afcd-ef4a59c159d0 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.356104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c42bcd-2a34-43d1-afcd-ef4a59c159d0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.356373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c42bcd-2a34-43d1-afcd-ef4a59c159d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.358617 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6530f1-07de-4e66-a368-1551d0a529a0 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:36.374297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a6c42bcd-2a34-43d1-afcd-ef4a59c159d0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:36.375515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a6c42bcd-2a34-43d1-afcd-ef4a59c159d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:36.394895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c71394-ea67-4695-975e-fc3c79685967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:36.395464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c71394-ea67-4695-975e-fc3c79685967 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.395742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c71394-ea67-4695-975e-fc3c79685967 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.396014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c71394-ea67-4695-975e-fc3c79685967 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.396619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c71394-ea67-4695-975e-fc3c79685967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:36.398105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c71394-ea67-4695-975e-fc3c79685967 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:36.410265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 377/757] 10.222.0.58 () {56 vars in 1087 bytes} [Fri Oct 28 18:54:36 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 18:54:36.463178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 381/758] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:36 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:36.518699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c71394-ea67-4695-975e-fc3c79685967 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:36.632472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c71394-ea67-4695-975e-fc3c79685967 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:36.644601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:36.670792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b63ae-313c-46d4-a3e1-9de0e290c96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:36.671373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b63ae-313c-46d4-a3e1-9de0e290c96b None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.671712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b63ae-313c-46d4-a3e1-9de0e290c96b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.671990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b63ae-313c-46d4-a3e1-9de0e290c96b None admin] PATH_INFO: `/v3/services/image` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.672573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b63ae-313c-46d4-a3e1-9de0e290c96b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:36.673940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b63ae-313c-46d4-a3e1-9de0e290c96b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-1c9b63ae-313c-46d4-a3e1-9de0e290c96b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:36.683271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:36.684261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:36.685087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 378/759] 10.222.0.58 () {56 vars in 1105 bytes} [Fri Oct 28 18:54:36 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:36.690285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c42bcd-2a34-43d1-afcd-ef4a59c159d0 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:36.702315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:36.737641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b63ae-313c-46d4-a3e1-9de0e290c96b None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:36.808165 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c42bcd-2a34-43d1-afcd-ef4a59c159d0 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:36.846016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30827b-63ef-4109-8eb8-a6f55dd28a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:36.846622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30827b-63ef-4109-8eb8-a6f55dd28a2d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.846898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30827b-63ef-4109-8eb8-a6f55dd28a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.847207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30827b-63ef-4109-8eb8-a6f55dd28a2d None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.847832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30827b-63ef-4109-8eb8-a6f55dd28a2d None admin] RBAC: Adding query filter params (name=image) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:36.848132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30827b-63ef-4109-8eb8-a6f55dd28a2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:36.849494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30827b-63ef-4109-8eb8-a6f55dd28a2d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:36.855261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 382/760] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:36.861204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30827b-63ef-4109-8eb8-a6f55dd28a2d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:36.867459 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b63ae-313c-46d4-a3e1-9de0e290c96b None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:36.903700 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3058a-d396-4642-bf97-ca834a1ab89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:36.904373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3058a-d396-4642-bf97-ca834a1ab89a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:36.904651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3058a-d396-4642-bf97-ca834a1ab89a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:36.904924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3058a-d396-4642-bf97-ca834a1ab89a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:36.905548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3058a-d396-4642-bf97-ca834a1ab89a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:36.906942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3058a-d396-4642-bf97-ca834a1ab89a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-32f3058a-d396-4642-bf97-ca834a1ab89a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:36.916214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:36.917381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 18:54:36.918300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:36.922523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 379/761] 10.222.0.58 () {56 vars in 1126 bytes} [Fri Oct 28 18:54:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:36.923781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3058a-d396-4642-bf97-ca834a1ab89a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:36.954549 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:36.981226 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30827b-63ef-4109-8eb8-a6f55dd28a2d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:36.982791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:37.017502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b5ae9-b348-4200-89a1-3f9b41b2cf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:37.018110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b5ae9-b348-4200-89a1-3f9b41b2cf7d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:37.018645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b5ae9-b348-4200-89a1-3f9b41b2cf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:37.018915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b5ae9-b348-4200-89a1-3f9b41b2cf7d None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:37.019662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b5ae9-b348-4200-89a1-3f9b41b2cf7d None admin] RBAC: Adding query filter params (type=image) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:37.019967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b5ae9-b348-4200-89a1-3f9b41b2cf7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:37.021320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b5ae9-b348-4200-89a1-3f9b41b2cf7d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:37.031157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 383/762] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:37.033752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b5ae9-b348-4200-89a1-3f9b41b2cf7d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:37.070239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3058a-d396-4642-bf97-ca834a1ab89a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:37.121025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3e7ea-c330-4509-af98-200e39444724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:37.121645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3e7ea-c330-4509-af98-200e39444724 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:37.122021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3e7ea-c330-4509-af98-200e39444724 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:37.122299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3e7ea-c330-4509-af98-200e39444724 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:37.122935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3e7ea-c330-4509-af98-200e39444724 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:37.123272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3e7ea-c330-4509-af98-200e39444724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:37.124897 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3e7ea-c330-4509-af98-200e39444724 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:37.130627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 380/763] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:37.136784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3e7ea-c330-4509-af98-200e39444724 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:37.148483 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b5ae9-b348-4200-89a1-3f9b41b2cf7d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:37.186228 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14eb94-84fa-4b48-a8e3-bb5a004f3d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:37.186798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14eb94-84fa-4b48-a8e3-bb5a004f3d63 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:37.187115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14eb94-84fa-4b48-a8e3-bb5a004f3d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:37.187382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14eb94-84fa-4b48-a8e3-bb5a004f3d63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:37.188362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14eb94-84fa-4b48-a8e3-bb5a004f3d63 None admin] RBAC: Adding query filter params (interface=public, service_id=97c7370b50d44b53bee87cba7bf93955, region_id=RegionOne) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:37.189135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14eb94-84fa-4b48-a8e3-bb5a004f3d63 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:37.190761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14eb94-84fa-4b48-a8e3-bb5a004f3d63 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:37.196296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 384/764] 10.222.0.58 () {56 vars in 1251 bytes} [Fri Oct 28 18:54:37 2022] GET /identity/v3/endpoints?service_id=97c7370b50d44b53bee87cba7bf93955&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:37.197464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b907162-7016-4f0a-8018-425528087a2f None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:37.197739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b907162-7016-4f0a-8018-425528087a2f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:37.198003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b907162-7016-4f0a-8018-425528087a2f None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:37.199838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 385/765] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:37.203880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b907162-7016-4f0a-8018-425528087a2f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:37.206598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:37.251650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3e7ea-c330-4509-af98-200e39444724 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:37.253294 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:37.289279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e8f87-a27e-4d56-bb0f-20f6178a70f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:37.289877 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e8f87-a27e-4d56-bb0f-20f6178a70f2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:37.290151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e8f87-a27e-4d56-bb0f-20f6178a70f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:37.290778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e8f87-a27e-4d56-bb0f-20f6178a70f2 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:37.291489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e8f87-a27e-4d56-bb0f-20f6178a70f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:37.292148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e8f87-a27e-4d56-bb0f-20f6178a70f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:37.293764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e8f87-a27e-4d56-bb0f-20f6178a70f2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:37.304257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 381/766] 10.222.0.58 () {56 vars in 1122 bytes} [Fri Oct 28 18:54:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:37.304257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4fba-15de-49e3-b66b-5af4a74baebd None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:37.304257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4fba-15de-49e3-b66b-5af4a74baebd None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:37.304257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4fba-15de-49e3-b66b-5af4a74baebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:37.314201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b907162-7016-4f0a-8018-425528087a2f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:37.324846 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-c23e4fba-15de-49e3-b66b-5af4a74baebd None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:37.326613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c23e4fba-15de-49e3-b66b-5af4a74baebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:37.348570 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0eb3e4-9f74-40ab-891a-70ccbd58b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:37.349183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eb3e4-9f74-40ab-891a-70ccbd58b6ba None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:37.349447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eb3e4-9f74-40ab-891a-70ccbd58b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:37.349715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eb3e4-9f74-40ab-891a-70ccbd58b6ba None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:37.350399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0eb3e4-9f74-40ab-891a-70ccbd58b6ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:37.351783 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0eb3e4-9f74-40ab-891a-70ccbd58b6ba None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:37.358599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 386/767] 10.222.0.58 () {56 vars in 1087 bytes} [Fri Oct 28 18:54:37 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 18:54:37.364035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0eb3e4-9f74-40ab-891a-70ccbd58b6ba None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:37.413496 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 382/768] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:37 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:37.455475 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:37.490942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0eb3e4-9f74-40ab-891a-70ccbd58b6ba None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:37.529139 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62a59a-8cfb-40b1-b4a4-80e294c8c939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:37.529758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62a59a-8cfb-40b1-b4a4-80e294c8c939 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:37.530042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62a59a-8cfb-40b1-b4a4-80e294c8c939 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:37.530308 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62a59a-8cfb-40b1-b4a4-80e294c8c939 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:37.530990 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62a59a-8cfb-40b1-b4a4-80e294c8c939 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:37.532392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62a59a-8cfb-40b1-b4a4-80e294c8c939 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:37.553664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 387/769] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 18:54:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:37.704014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4fba-15de-49e3-b66b-5af4a74baebd None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:37.805587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:37.812158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4fba-15de-49e3-b66b-5af4a74baebd None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:37.853223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:37.853831 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:37.854116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:37.854392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:37.855680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:37.857161 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:37.859720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING keystone.server.flask.common [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 28 18:54:37.873789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.sql.core [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 28 18:54:37.873789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 28 18:54:37.873789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [parameters: {'user_id': '44252963199f49fd8c790781227353a5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 28 18:54:37.873789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81364) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result.read() Oct 28 18:54:37.881289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:54:37.882125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application yield resource Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.session.rollback() Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.force_reraise() Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise self.value Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application session.commit() Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 28 18:54:37.883310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._prepare_impl() Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.session.flush() Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._flush(objects) Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application compat.raise_( Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application flush_context.execute() Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rec.execute(self) Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 28 18:54:37.884740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:54:37.885836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result.read() Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:54:37.886857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [parameters: {'user_id': '44252963199f49fd8c790781227353a5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:37.888975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 28 18:54:37.890695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 28 18:54:37.895472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 28 18:54:37.895472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 28 18:54:37.895472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 28 18:54:37.895472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 28 18:54:37.895472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 28 18:54:37.895472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:37.895472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 383/770] 10.222.0.58 () {60 vars in 1131 bytes} [Fri Oct 28 18:54:37 2022] POST /identity/v3/users => generated 160 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:37.895977 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62a59a-8cfb-40b1-b4a4-80e294c8c939 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:38.009006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62a59a-8cfb-40b1-b4a4-80e294c8c939 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:38.036234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:38.047039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30175ab-1ff6-4075-9924-e1b9abbc810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:38.047991 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30175ab-1ff6-4075-9924-e1b9abbc810e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:38.048474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30175ab-1ff6-4075-9924-e1b9abbc810e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:38.049055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30175ab-1ff6-4075-9924-e1b9abbc810e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:38.173430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30175ab-1ff6-4075-9924-e1b9abbc810e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:38.175412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30175ab-1ff6-4075-9924-e1b9abbc810e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-e30175ab-1ff6-4075-9924-e1b9abbc810e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:38.188117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:38.189069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 18:54:38.190004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:38.192546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 388/771] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:54:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:38.199328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:38.304025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca106-a055-41ee-a2cc-cee766275885 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:38.343660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff47348-6808-4726-93a7-ae5dd0802c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:38.344453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47348-6808-4726-93a7-ae5dd0802c9a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:38.344830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47348-6808-4726-93a7-ae5dd0802c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:38.345204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47348-6808-4726-93a7-ae5dd0802c9a None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:38.346010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47348-6808-4726-93a7-ae5dd0802c9a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:38.346418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47348-6808-4726-93a7-ae5dd0802c9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:38.348182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47348-6808-4726-93a7-ae5dd0802c9a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:38.391763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 384/772] 10.222.0.58 () {56 vars in 1101 bytes} [Fri Oct 28 18:54:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:38.687993 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81463ec-e257-4847-b751-379316c8077b None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:38.688203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81463ec-e257-4847-b751-379316c8077b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:38.688466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81463ec-e257-4847-b751-379316c8077b None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:38.689930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 389/773] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:38.695184 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2deab7-42de-44fd-935f-37493f68414a None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:38.695464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2deab7-42de-44fd-935f-37493f68414a None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:38.695735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2deab7-42de-44fd-935f-37493f68414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:38.713708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-1e2deab7-42de-44fd-935f-37493f68414a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:38.715250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1e2deab7-42de-44fd-935f-37493f68414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:38.722557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:38.789077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 385/774] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:38 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:39.058237 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81463ec-e257-4847-b751-379316c8077b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:39.163067 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81463ec-e257-4847-b751-379316c8077b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:39.197908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77565f36-94ac-483a-83a9-839d3e372d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:39.198559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77565f36-94ac-483a-83a9-839d3e372d2e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:39.198832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77565f36-94ac-483a-83a9-839d3e372d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:39.199125 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77565f36-94ac-483a-83a9-839d3e372d2e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:39.199701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77565f36-94ac-483a-83a9-839d3e372d2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:39.201067 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77565f36-94ac-483a-83a9-839d3e372d2e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-77565f36-94ac-483a-83a9-839d3e372d2e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:39.209834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:39.210791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 18:54:39.211955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:39.213091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 390/775] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:39.216774 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2deab7-42de-44fd-935f-37493f68414a None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:39.319463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2deab7-42de-44fd-935f-37493f68414a None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:39.352902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b90f4-1567-4e4c-bad4-8cb356c4ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:39.353507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b90f4-1567-4e4c-bad4-8cb356c4ed24 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:39.353780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b90f4-1567-4e4c-bad4-8cb356c4ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:39.354050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b90f4-1567-4e4c-bad4-8cb356c4ed24 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:39.355259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b90f4-1567-4e4c-bad4-8cb356c4ed24 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:39.355565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b90f4-1567-4e4c-bad4-8cb356c4ed24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:39.356929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b90f4-1567-4e4c-bad4-8cb356c4ed24 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:39.366691 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 386/776] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:39.367931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77565f36-94ac-483a-83a9-839d3e372d2e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:39.419568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:39.470418 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77565f36-94ac-483a-83a9-839d3e372d2e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:39.504126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aae7ef-f56c-44e0-bef3-19d02c7f927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:39.504724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aae7ef-f56c-44e0-bef3-19d02c7f927f None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:39.504995 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aae7ef-f56c-44e0-bef3-19d02c7f927f None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:39.505255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aae7ef-f56c-44e0-bef3-19d02c7f927f None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:39.505873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aae7ef-f56c-44e0-bef3-19d02c7f927f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:39.506167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aae7ef-f56c-44e0-bef3-19d02c7f927f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:39.507551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aae7ef-f56c-44e0-bef3-19d02c7f927f None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:39.517516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 391/777] 10.222.0.58 () {56 vars in 1112 bytes} [Fri Oct 28 18:54:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:39.519409 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b90f4-1567-4e4c-bad4-8cb356c4ed24 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:39.529430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:39.624112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b90f4-1567-4e4c-bad4-8cb356c4ed24 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:39.639970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26f53c-01d4-45d5-a860-5f15c5b918c0 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:39.640253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26f53c-01d4-45d5-a860-5f15c5b918c0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:39.640524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26f53c-01d4-45d5-a860-5f15c5b918c0 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:39.641971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 392/778] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:39.646921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557ca1b-5656-42f9-aad5-96d87f445409 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:39.647231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557ca1b-5656-42f9-aad5-96d87f445409 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:39.647504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557ca1b-5656-42f9-aad5-96d87f445409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:39.657158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ff14a-8d6b-42ab-8c43-f140bcfc4adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:39.657765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ff14a-8d6b-42ab-8c43-f140bcfc4adb None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:39.658045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ff14a-8d6b-42ab-8c43-f140bcfc4adb None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:39.658316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ff14a-8d6b-42ab-8c43-f140bcfc4adb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:39.659004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ff14a-8d6b-42ab-8c43-f140bcfc4adb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:39.660412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ff14a-8d6b-42ab-8c43-f140bcfc4adb None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:39.670712 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f557ca1b-5656-42f9-aad5-96d87f445409 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:39.672056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f557ca1b-5656-42f9-aad5-96d87f445409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:39.693373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 387/779] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 18:54:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:54:39.771027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 393/780] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:39.797962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d07fef-893b-4ed6-8b6d-f3683a78820a None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:39.798563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d07fef-893b-4ed6-8b6d-f3683a78820a None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:39.798701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d07fef-893b-4ed6-8b6d-f3683a78820a None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:39.800199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 388/781] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:39.806042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84646a22-f857-44ef-8286-7857f3346388 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:39.806378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84646a22-f857-44ef-8286-7857f3346388 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:39.806591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84646a22-f857-44ef-8286-7857f3346388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:39.825930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-84646a22-f857-44ef-8286-7857f3346388 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:39.827161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-84646a22-f857-44ef-8286-7857f3346388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:39.868359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:39.903335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 394/782] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:40.044037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d07fef-893b-4ed6-8b6d-f3683a78820a None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:40.107057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:40.148816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d07fef-893b-4ed6-8b6d-f3683a78820a None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:40.167735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84646a22-f857-44ef-8286-7857f3346388 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:40.185798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffda4af-fa49-4ac5-b219-48b53ca2a4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:40.186422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffda4af-fa49-4ac5-b219-48b53ca2a4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:40.186693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffda4af-fa49-4ac5-b219-48b53ca2a4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:40.186957 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffda4af-fa49-4ac5-b219-48b53ca2a4e6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:40.187570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffda4af-fa49-4ac5-b219-48b53ca2a4e6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:40.188928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffda4af-fa49-4ac5-b219-48b53ca2a4e6 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-3ffda4af-fa49-4ac5-b219-48b53ca2a4e6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:40.197717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:40.203079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:40.204223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 389/783] 10.222.0.58 () {56 vars in 1105 bytes} [Fri Oct 28 18:54:40 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:40.206702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffda4af-fa49-4ac5-b219-48b53ca2a4e6 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:40.276585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84646a22-f857-44ef-8286-7857f3346388 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:40.310174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f472aa1-c802-4867-b84e-9a09fe0c0544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:40.310795 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f472aa1-c802-4867-b84e-9a09fe0c0544 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:40.311097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f472aa1-c802-4867-b84e-9a09fe0c0544 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:40.311364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f472aa1-c802-4867-b84e-9a09fe0c0544 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:40.312435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f472aa1-c802-4867-b84e-9a09fe0c0544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:40.314114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f472aa1-c802-4867-b84e-9a09fe0c0544 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:40.315805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 395/784] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:40.317511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffda4af-fa49-4ac5-b219-48b53ca2a4e6 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:40.321334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f472aa1-c802-4867-b84e-9a09fe0c0544 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:40.345160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:40.354734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c5317-908d-4679-9019-9b2e080e76ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:40.355356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c5317-908d-4679-9019-9b2e080e76ee None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:40.355631 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c5317-908d-4679-9019-9b2e080e76ee None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:40.355899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c5317-908d-4679-9019-9b2e080e76ee None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:40.356514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c5317-908d-4679-9019-9b2e080e76ee None admin] RBAC: Adding query filter params (name=image) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:40.356809 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c5317-908d-4679-9019-9b2e080e76ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:40.358182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c5317-908d-4679-9019-9b2e080e76ee None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:40.361883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:40.362917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 390/785] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:40.368727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c5317-908d-4679-9019-9b2e080e76ee None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:40.429965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f472aa1-c802-4867-b84e-9a09fe0c0544 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:40.471144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e590e59-046d-4397-8b3f-92d77fbf6999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:40.471756 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e590e59-046d-4397-8b3f-92d77fbf6999 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:40.472089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e590e59-046d-4397-8b3f-92d77fbf6999 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:40.472359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e590e59-046d-4397-8b3f-92d77fbf6999 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:40.472464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c5317-908d-4679-9019-9b2e080e76ee None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:40.473061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e590e59-046d-4397-8b3f-92d77fbf6999 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:40.475696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e590e59-046d-4397-8b3f-92d77fbf6999 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:40.507895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 396/786] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:54:40 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:54:40.509004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54af81-b13b-4a55-8a91-38a3c4ffe84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:40.509786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54af81-b13b-4a55-8a91-38a3c4ffe84d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:40.510066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54af81-b13b-4a55-8a91-38a3c4ffe84d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:40.510334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54af81-b13b-4a55-8a91-38a3c4ffe84d None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:40.511559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54af81-b13b-4a55-8a91-38a3c4ffe84d None admin] RBAC: Adding query filter params (type=image) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:40.511861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54af81-b13b-4a55-8a91-38a3c4ffe84d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:40.513255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54af81-b13b-4a55-8a91-38a3c4ffe84d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:40.519143 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 391/787] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:40.525235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e590e59-046d-4397-8b3f-92d77fbf6999 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:40.621657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:40.633025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e590e59-046d-4397-8b3f-92d77fbf6999 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:40.672221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16f388-0209-4dc2-9587-bd0d7200d89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:40.672842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16f388-0209-4dc2-9587-bd0d7200d89f None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:40.673114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16f388-0209-4dc2-9587-bd0d7200d89f None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:40.673378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16f388-0209-4dc2-9587-bd0d7200d89f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:40.674043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16f388-0209-4dc2-9587-bd0d7200d89f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:40.675450 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16f388-0209-4dc2-9587-bd0d7200d89f None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:40.697246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 397/788] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 18:54:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:41.922779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51e58a-a2cc-4d8c-96cf-15341237457c None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:41.923361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51e58a-a2cc-4d8c-96cf-15341237457c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:41.923361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51e58a-a2cc-4d8c-96cf-15341237457c None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:41.924717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 392/789] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:41.929896 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b46fd5-b3c1-446b-8a09-d68d606a3a61 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:41.930311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b46fd5-b3c1-446b-8a09-d68d606a3a61 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:41.930503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b46fd5-b3c1-446b-8a09-d68d606a3a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:41.949786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-b7b46fd5-b3c1-446b-8a09-d68d606a3a61 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:41.950955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b7b46fd5-b3c1-446b-8a09-d68d606a3a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:42.023661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 398/790] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:42.287732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51e58a-a2cc-4d8c-96cf-15341237457c None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:42.346597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:42.395885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51e58a-a2cc-4d8c-96cf-15341237457c None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:42.431829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bb8c9-f2bf-46f4-ad0d-4d9bc5322322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:42.432456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bb8c9-f2bf-46f4-ad0d-4d9bc5322322 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:42.433277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bb8c9-f2bf-46f4-ad0d-4d9bc5322322 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:42.433545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bb8c9-f2bf-46f4-ad0d-4d9bc5322322 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:42.443895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bb8c9-f2bf-46f4-ad0d-4d9bc5322322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-f40bb8c9-f2bf-46f4-ad0d-4d9bc5322322 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:42.448906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:42.449812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:42.451710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 393/791] 10.222.0.58 () {56 vars in 1120 bytes} [Fri Oct 28 18:54:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:42.456470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b46fd5-b3c1-446b-8a09-d68d606a3a61 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:42.526910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:42.558375 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b46fd5-b3c1-446b-8a09-d68d606a3a61 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:42.592675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382180a8-73e4-4dd6-ae80-037011a196ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:42.593278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382180a8-73e4-4dd6-ae80-037011a196ca None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:42.593550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382180a8-73e4-4dd6-ae80-037011a196ca None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:42.593815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382180a8-73e4-4dd6-ae80-037011a196ca None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:42.594471 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382180a8-73e4-4dd6-ae80-037011a196ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:42.594768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382180a8-73e4-4dd6-ae80-037011a196ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:42.596152 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382180a8-73e4-4dd6-ae80-037011a196ca None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:42.605529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 399/792] 10.222.0.58 () {56 vars in 1115 bytes} [Fri Oct 28 18:54:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:42.611416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bb8c9-f2bf-46f4-ad0d-4d9bc5322322 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:42.715158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bb8c9-f2bf-46f4-ad0d-4d9bc5322322 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:42.730563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea32023-b446-4cc5-9513-516fc2b65e4d None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:42.730563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea32023-b446-4cc5-9513-516fc2b65e4d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:42.730563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea32023-b446-4cc5-9513-516fc2b65e4d None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:42.730563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 400/793] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:54:42.736201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d432b90-3b25-4431-ab90-370fae6300b9 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:42.737118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d432b90-3b25-4431-ab90-370fae6300b9 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:42.737118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d432b90-3b25-4431-ab90-370fae6300b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:42.737285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:42.757164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-4d432b90-3b25-4431-ab90-370fae6300b9 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:42.758636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4d432b90-3b25-4431-ab90-370fae6300b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:42.766314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909c157-df40-49f3-976a-843e7f1d0e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:42.766936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909c157-df40-49f3-976a-843e7f1d0e5a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:42.767240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909c157-df40-49f3-976a-843e7f1d0e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:42.767508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909c157-df40-49f3-976a-843e7f1d0e5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:42.779477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909c157-df40-49f3-976a-843e7f1d0e5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:42.780930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909c157-df40-49f3-976a-843e7f1d0e5a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-2909c157-df40-49f3-976a-843e7f1d0e5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:42.795613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:42.796589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:42.797486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:42.797486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:42.797486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:42.797486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 394/794] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:42.802354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909c157-df40-49f3-976a-843e7f1d0e5a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:42.831933 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 401/795] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:42.847057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e60660-1e89-41d7-8250-fefbd7d34295 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:42.847569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e60660-1e89-41d7-8250-fefbd7d34295 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:42.847838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e60660-1e89-41d7-8250-fefbd7d34295 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:42.849286 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 402/796] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:54:42.854080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aeaaa-dfef-4209-b11e-677b895da7d0 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:42.855171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aeaaa-dfef-4209-b11e-677b895da7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:42.855448 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aeaaa-dfef-4209-b11e-677b895da7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:42.873625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-924aeaaa-dfef-4209-b11e-677b895da7d0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:42.874785 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-924aeaaa-dfef-4209-b11e-677b895da7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:42.908801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909c157-df40-49f3-976a-843e7f1d0e5a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:42.930694 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:42.943041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 403/797] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:42.947804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c2b6d-c554-47d4-99b0-4558491ccf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:42.948929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c2b6d-c554-47d4-99b0-4558491ccf72 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:42.949205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c2b6d-c554-47d4-99b0-4558491ccf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:42.949473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c2b6d-c554-47d4-99b0-4558491ccf72 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:42.950105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c2b6d-c554-47d4-99b0-4558491ccf72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:42.950428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c2b6d-c554-47d4-99b0-4558491ccf72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:42.951806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c2b6d-c554-47d4-99b0-4558491ccf72 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:42.963622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 395/798] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:42.969686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aeaaa-dfef-4209-b11e-677b895da7d0 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.065105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aeaaa-dfef-4209-b11e-677b895da7d0 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.099219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8a8c1-e77c-49ca-9d40-f87d21eb5d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.099772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8a8c1-e77c-49ca-9d40-f87d21eb5d0f None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.100046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8a8c1-e77c-49ca-9d40-f87d21eb5d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.100313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8a8c1-e77c-49ca-9d40-f87d21eb5d0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.107251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8a8c1-e77c-49ca-9d40-f87d21eb5d0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.108681 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8a8c1-e77c-49ca-9d40-f87d21eb5d0f None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.112098 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c2b6d-c554-47d4-99b0-4558491ccf72 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-2ba8a8c1-e77c-49ca-9d40-f87d21eb5d0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:43.120427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:43.121357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:43.122320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 404/799] 10.222.0.58 () {56 vars in 1093 bytes} [Fri Oct 28 18:54:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:43.127135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8a8c1-e77c-49ca-9d40-f87d21eb5d0f None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.143064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:43.184727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:43.218195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c2b6d-c554-47d4-99b0-4558491ccf72 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.233436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8a8c1-e77c-49ca-9d40-f87d21eb5d0f None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.252055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cdf9e-93f6-443e-964f-5e999749244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.252676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cdf9e-93f6-443e-964f-5e999749244a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.252948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cdf9e-93f6-443e-964f-5e999749244a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.253215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cdf9e-93f6-443e-964f-5e999749244a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.254300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cdf9e-93f6-443e-964f-5e999749244a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.255746 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cdf9e-93f6-443e-964f-5e999749244a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.257295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 396/800] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:43.258016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cdf9e-93f6-443e-964f-5e999749244a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.266635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ce68d-b576-48a2-94c6-7e3c53ccbf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.267258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ce68d-b576-48a2-94c6-7e3c53ccbf50 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.267532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ce68d-b576-48a2-94c6-7e3c53ccbf50 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.267898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ce68d-b576-48a2-94c6-7e3c53ccbf50 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.268946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ce68d-b576-48a2-94c6-7e3c53ccbf50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:43.269246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ce68d-b576-48a2-94c6-7e3c53ccbf50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.270776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ce68d-b576-48a2-94c6-7e3c53ccbf50 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.287382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 405/801] 10.222.0.58 () {56 vars in 1133 bytes} [Fri Oct 28 18:54:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:43.288138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ce68d-b576-48a2-94c6-7e3c53ccbf50 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.350713 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:43.357344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cdf9e-93f6-443e-964f-5e999749244a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.385126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ce68d-b576-48a2-94c6-7e3c53ccbf50 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.391874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.392470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.392750 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.393021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.393932 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.395605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.399219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:43.403648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.sql.core [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 28 18:54:43.403648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 28 18:54:43.403648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [parameters: {'id': '26974d8a645e4a83922f74a8d4c3c009', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 28 18:54:43.403648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81364) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:54:43.410710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result.read() Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 28 18:54:43.411620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application yield resource Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application next(self.gen) Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.session.rollback() Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.force_reraise() Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise self.value Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application session.commit() Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 28 18:54:43.415981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._prepare_impl() Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.session.flush() Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._flush(objects) Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application compat.raise_( Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application flush_context.execute() Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rec.execute(self) Oct 28 18:54:43.417084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 28 18:54:43.417978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application conn.query(q) Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application result.read() Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 18:54:43.418725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [parameters: {'id': '26974d8a645e4a83922f74a8d4c3c009', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 28 18:54:43.419654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:43.420431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 397/802] 10.222.0.58 () {60 vars in 1137 bytes} [Fri Oct 28 18:54:43 2022] POST /identity/v3/domains => generated 190 bytes in 154 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:43.421304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.425699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435c8c2-ff5f-42cf-b28c-560887ee95b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.426328 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435c8c2-ff5f-42cf-b28c-560887ee95b6 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.426596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435c8c2-ff5f-42cf-b28c-560887ee95b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.426860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435c8c2-ff5f-42cf-b28c-560887ee95b6 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.427529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435c8c2-ff5f-42cf-b28c-560887ee95b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.428932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435c8c2-ff5f-42cf-b28c-560887ee95b6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.455038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 406/803] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:54:43 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:43.455038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435c8c2-ff5f-42cf-b28c-560887ee95b6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.524168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd444e-aa89-45db-b01e-99671a6a75ee None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.562398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ac1c9-c10e-41c3-9625-df0dad15b66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.563080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ac1c9-c10e-41c3-9625-df0dad15b66d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.563210 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435c8c2-ff5f-42cf-b28c-560887ee95b6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.563383 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ac1c9-c10e-41c3-9625-df0dad15b66d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.563619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ac1c9-c10e-41c3-9625-df0dad15b66d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.575478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ac1c9-c10e-41c3-9625-df0dad15b66d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.577030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ac1c9-c10e-41c3-9625-df0dad15b66d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-fe4ac1c9-c10e-41c3-9625-df0dad15b66d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:43.592990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:43.594027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:43.594931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:43.594931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:43.594931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:43.594931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 398/804] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:43.600283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ac1c9-c10e-41c3-9625-df0dad15b66d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.603226 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46de8b-f48d-4ce7-90ab-e778363c0632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.603817 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46de8b-f48d-4ce7-90ab-e778363c0632 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.604078 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46de8b-f48d-4ce7-90ab-e778363c0632 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.604930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46de8b-f48d-4ce7-90ab-e778363c0632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.614853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:43.617256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46de8b-f48d-4ce7-90ab-e778363c0632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.619516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46de8b-f48d-4ce7-90ab-e778363c0632 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-8f46de8b-f48d-4ce7-90ab-e778363c0632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:43.636136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:43.637223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:43.638102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:43.638102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:43.638102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:43.638102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 407/805] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:43.643743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46de8b-f48d-4ce7-90ab-e778363c0632 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.668106 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:43.722927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ac1c9-c10e-41c3-9625-df0dad15b66d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.759240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46de8b-f48d-4ce7-90ab-e778363c0632 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.762578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eedba9-f4e4-4bd8-9571-093485d9575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.763208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eedba9-f4e4-4bd8-9571-093485d9575e None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.763484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eedba9-f4e4-4bd8-9571-093485d9575e None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.763752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eedba9-f4e4-4bd8-9571-093485d9575e None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.764392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eedba9-f4e4-4bd8-9571-093485d9575e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:43.764693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eedba9-f4e4-4bd8-9571-093485d9575e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.766048 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eedba9-f4e4-4bd8-9571-093485d9575e None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.778375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 399/806] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:43.783794 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eedba9-f4e4-4bd8-9571-093485d9575e None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:43.798573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ff142-5d93-4d58-a9f8-b40e04949801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.799358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ff142-5d93-4d58-a9f8-b40e04949801 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.799494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ff142-5d93-4d58-a9f8-b40e04949801 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.799742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ff142-5d93-4d58-a9f8-b40e04949801 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.800383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ff142-5d93-4d58-a9f8-b40e04949801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:43.800683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ff142-5d93-4d58-a9f8-b40e04949801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.802128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ff142-5d93-4d58-a9f8-b40e04949801 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.815080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 408/807] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:43.864749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:43.900796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eedba9-f4e4-4bd8-9571-093485d9575e None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:43.939186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579724d-3f2e-4853-9647-1fbaa614a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:43.939838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579724d-3f2e-4853-9647-1fbaa614a2af None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:43.940119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579724d-3f2e-4853-9647-1fbaa614a2af None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:43.940393 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579724d-3f2e-4853-9647-1fbaa614a2af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:43.948022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579724d-3f2e-4853-9647-1fbaa614a2af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:43.949548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579724d-3f2e-4853-9647-1fbaa614a2af None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-a579724d-3f2e-4853-9647-1fbaa614a2af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:43.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:43.962958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:43.964236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:43.964236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:43.964236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:43.964236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:43.964236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:43.964236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:43.964236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:43.964236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:43.964236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 400/808] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:43.969232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ff142-5d93-4d58-a9f8-b40e04949801 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:44.056448 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:44.077168 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ff142-5d93-4d58-a9f8-b40e04949801 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:44.114744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d490c4-12fc-444c-a5f0-84c7ed818211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:44.115437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d490c4-12fc-444c-a5f0-84c7ed818211 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:44.115666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d490c4-12fc-444c-a5f0-84c7ed818211 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:44.115930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d490c4-12fc-444c-a5f0-84c7ed818211 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:44.116613 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d490c4-12fc-444c-a5f0-84c7ed818211 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:44.116910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d490c4-12fc-444c-a5f0-84c7ed818211 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:44.118319 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d490c4-12fc-444c-a5f0-84c7ed818211 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:44.130391 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 409/809] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:44.136909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579724d-3f2e-4853-9647-1fbaa614a2af None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:44.235959 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579724d-3f2e-4853-9647-1fbaa614a2af None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:44.263250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:44.270004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecbfb9-a167-4db3-811f-c24ea406096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:44.270655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecbfb9-a167-4db3-811f-c24ea406096b None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:44.270933 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecbfb9-a167-4db3-811f-c24ea406096b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:44.271234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecbfb9-a167-4db3-811f-c24ea406096b None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a758611cf8d94e8ca8aad6a75bef5e34/roles/2604cfbe6860499e9e894574ea9c0665` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:44.273491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecbfb9-a167-4db3-811f-c24ea406096b None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=a758611cf8d94e8ca8aad6a75bef5e34, role_id=2604cfbe6860499e9e894574ea9c0665)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:44.274942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecbfb9-a167-4db3-811f-c24ea406096b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:44.287662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 401/810] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:54:44 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/a758611cf8d94e8ca8aad6a75bef5e34/roles/2604cfbe6860499e9e894574ea9c0665 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:54:45.633930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4a869-aac8-4a27-ae80-159acf14e8bd None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:45.634495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4a869-aac8-4a27-ae80-159acf14e8bd None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:45.634495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4a869-aac8-4a27-ae80-159acf14e8bd None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:45.635834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 410/811] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:45.641194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72425986-16fd-478b-a527-960c9be9dbc9 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:45.641596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72425986-16fd-478b-a527-960c9be9dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:45.641851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72425986-16fd-478b-a527-960c9be9dbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:45.660424 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-72425986-16fd-478b-a527-960c9be9dbc9 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:45.661607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-72425986-16fd-478b-a527-960c9be9dbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:45.728540 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 402/812] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:45.994771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4a869-aac8-4a27-ae80-159acf14e8bd None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:46.091741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4a869-aac8-4a27-ae80-159acf14e8bd None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:46.123109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b72a9-2dac-4b03-8df4-65ee972dd796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:46.123721 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b72a9-2dac-4b03-8df4-65ee972dd796 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:46.123992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b72a9-2dac-4b03-8df4-65ee972dd796 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:46.124256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b72a9-2dac-4b03-8df4-65ee972dd796 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:46.131413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:46.134772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b72a9-2dac-4b03-8df4-65ee972dd796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-7d6b72a9-2dac-4b03-8df4-65ee972dd796 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:46.140050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:46.141135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:46.141987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 411/813] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:54:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:46.147373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72425986-16fd-478b-a527-960c9be9dbc9 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:46.242523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72425986-16fd-478b-a527-960c9be9dbc9 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:46.262809 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:46.277506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57138443-851b-4abb-8f51-8e3063332a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:46.278106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57138443-851b-4abb-8f51-8e3063332a62 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:46.278384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57138443-851b-4abb-8f51-8e3063332a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:46.278654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57138443-851b-4abb-8f51-8e3063332a62 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:46.279344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57138443-851b-4abb-8f51-8e3063332a62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:46.279648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57138443-851b-4abb-8f51-8e3063332a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:46.281006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57138443-851b-4abb-8f51-8e3063332a62 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:46.289745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 403/814] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:54:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:46.295880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b72a9-2dac-4b03-8df4-65ee972dd796 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:46.397211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b72a9-2dac-4b03-8df4-65ee972dd796 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:46.412064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694cf4-a117-4570-98f6-ef0ba9dbe14b None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:46.412350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694cf4-a117-4570-98f6-ef0ba9dbe14b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:46.412623 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694cf4-a117-4570-98f6-ef0ba9dbe14b None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:46.413940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 404/815] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:54:46.418806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f6fd3-e2e1-4942-a037-6dde715eb7af None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:46.419120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f6fd3-e2e1-4942-a037-6dde715eb7af None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:46.419393 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f6fd3-e2e1-4942-a037-6dde715eb7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:46.430158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab8b5b-9090-4f86-93e5-c3ea62d55cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:46.430780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab8b5b-9090-4f86-93e5-c3ea62d55cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:46.431093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab8b5b-9090-4f86-93e5-c3ea62d55cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:46.431367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab8b5b-9090-4f86-93e5-c3ea62d55cd8 None admin] PATH_INFO: `/v3/users/dd7eee59c750497a8f83d7c078736619` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:46.437488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-db1f6fd3-e2e1-4942-a037-6dde715eb7af None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:46.438674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-db1f6fd3-e2e1-4942-a037-6dde715eb7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:46.444689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab8b5b-9090-4f86-93e5-c3ea62d55cd8 None admin] RBAC: Authorizing `identity:get_user(user_id=dd7eee59c750497a8f83d7c078736619)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:46.446105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab8b5b-9090-4f86-93e5-c3ea62d55cd8 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:46.447886 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 412/816] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:54:46 2022] GET /identity/v3/users/dd7eee59c750497a8f83d7c078736619 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:46.453853 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab8b5b-9090-4f86-93e5-c3ea62d55cd8 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:46.506062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 405/817] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:54:46.509303 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:46.549373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab8b5b-9090-4f86-93e5-c3ea62d55cd8 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:46.582183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4069bc-b3ca-4e12-891f-7afa22e72664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:46.582809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4069bc-b3ca-4e12-891f-7afa22e72664 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:46.583116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4069bc-b3ca-4e12-891f-7afa22e72664 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:46.583389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4069bc-b3ca-4e12-891f-7afa22e72664 None admin] PATH_INFO: `/v3/projects/15e797a01bfd43da9e2a7368a3b17749` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:46.584494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4069bc-b3ca-4e12-891f-7afa22e72664 None admin] RBAC: Authorizing `identity:get_project(project_id=15e797a01bfd43da9e2a7368a3b17749)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:46.585911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4069bc-b3ca-4e12-891f-7afa22e72664 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:46.587542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 413/818] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:54:46 2022] GET /identity/v3/projects/15e797a01bfd43da9e2a7368a3b17749 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:46.593434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f6fd3-e2e1-4942-a037-6dde715eb7af None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:46.689182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f6fd3-e2e1-4942-a037-6dde715eb7af None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:46.701673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:46.721794 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9ccc9-8f77-4655-962a-a3b77a16c584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:46.722432 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9ccc9-8f77-4655-962a-a3b77a16c584 None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:46.722706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9ccc9-8f77-4655-962a-a3b77a16c584 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:46.722972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9ccc9-8f77-4655-962a-a3b77a16c584 None admin] PATH_INFO: `/v3/projects/15e797a01bfd43da9e2a7368a3b17749/users/dd7eee59c750497a8f83d7c078736619/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:46.724832 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9ccc9-8f77-4655-962a-a3b77a16c584 None admin] RBAC: Authorizing `identity:create_grant(project_id=15e797a01bfd43da9e2a7368a3b17749, user_id=dd7eee59c750497a8f83d7c078736619, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:46.726243 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9ccc9-8f77-4655-962a-a3b77a16c584 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:46.740884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 406/819] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:54:46 2022] PUT /identity/v3/projects/15e797a01bfd43da9e2a7368a3b17749/users/dd7eee59c750497a8f83d7c078736619/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:54:46.775330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4069bc-b3ca-4e12-891f-7afa22e72664 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:46.880163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4069bc-b3ca-4e12-891f-7afa22e72664 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:46.907502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:46.917451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30dd64-24cd-4fab-a7f8-1e2f01229925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:46.918081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30dd64-24cd-4fab-a7f8-1e2f01229925 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:46.918363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30dd64-24cd-4fab-a7f8-1e2f01229925 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:46.918636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30dd64-24cd-4fab-a7f8-1e2f01229925 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:46.929035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30dd64-24cd-4fab-a7f8-1e2f01229925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-6f30dd64-24cd-4fab-a7f8-1e2f01229925 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:46.934389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:46.935433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:46.936255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 414/820] 10.222.0.58 () {56 vars in 1120 bytes} [Fri Oct 28 18:54:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:46.941634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9ccc9-8f77-4655-962a-a3b77a16c584 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:47.040136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9ccc9-8f77-4655-962a-a3b77a16c584 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:47.074060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eff4b7-14e4-4969-83a3-e07944929b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:47.074654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eff4b7-14e4-4969-83a3-e07944929b47 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:47.074929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eff4b7-14e4-4969-83a3-e07944929b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:47.075222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eff4b7-14e4-4969-83a3-e07944929b47 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:47.075880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eff4b7-14e4-4969-83a3-e07944929b47 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:47.076179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eff4b7-14e4-4969-83a3-e07944929b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:47.077531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eff4b7-14e4-4969-83a3-e07944929b47 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:47.085658 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 407/821] 10.222.0.58 () {56 vars in 1115 bytes} [Fri Oct 28 18:54:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:47.092224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30dd64-24cd-4fab-a7f8-1e2f01229925 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:47.189044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30dd64-24cd-4fab-a7f8-1e2f01229925 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:47.221620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdabe41-744e-4aa4-8a5e-908d24c3ebee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:47.222238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdabe41-744e-4aa4-8a5e-908d24c3ebee None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:47.222515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdabe41-744e-4aa4-8a5e-908d24c3ebee None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:47.222784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdabe41-744e-4aa4-8a5e-908d24c3ebee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:47.232920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdabe41-744e-4aa4-8a5e-908d24c3ebee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:47.234351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdabe41-744e-4aa4-8a5e-908d24c3ebee None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-3bdabe41-744e-4aa4-8a5e-908d24c3ebee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:47.248515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:47.249595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:47.250630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:47.250630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:47.250630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:47.250630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 415/822] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:47.255414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eff4b7-14e4-4969-83a3-e07944929b47 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:47.289963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:47.356469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eff4b7-14e4-4969-83a3-e07944929b47 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:47.389983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268635dd-e929-4cc7-bd44-7add85fb67e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:47.390582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268635dd-e929-4cc7-bd44-7add85fb67e5 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:47.390858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268635dd-e929-4cc7-bd44-7add85fb67e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:47.391158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268635dd-e929-4cc7-bd44-7add85fb67e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:47.391841 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268635dd-e929-4cc7-bd44-7add85fb67e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:47.392149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268635dd-e929-4cc7-bd44-7add85fb67e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:47.393763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268635dd-e929-4cc7-bd44-7add85fb67e5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:47.405216 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 408/823] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:47.410926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdabe41-744e-4aa4-8a5e-908d24c3ebee None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:47.451767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:47.510025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdabe41-744e-4aa4-8a5e-908d24c3ebee None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:47.542629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5df98-803b-43f3-af53-a3029a2d67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:47.543269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5df98-803b-43f3-af53-a3029a2d67b8 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:47.543543 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5df98-803b-43f3-af53-a3029a2d67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:47.543805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5df98-803b-43f3-af53-a3029a2d67b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:47.550830 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5df98-803b-43f3-af53-a3029a2d67b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:47.552271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5df98-803b-43f3-af53-a3029a2d67b8 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-1bd5df98-803b-43f3-af53-a3029a2d67b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:54:47.563716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:47.564807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:47.570737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 416/824] 10.222.0.58 () {56 vars in 1093 bytes} [Fri Oct 28 18:54:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:47.571923 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268635dd-e929-4cc7-bd44-7add85fb67e5 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:47.668887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:47.676278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268635dd-e929-4cc7-bd44-7add85fb67e5 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:47.711367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17e19f-6cb2-41f7-858b-b7b459a756d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:47.711944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e19f-6cb2-41f7-858b-b7b459a756d1 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:47.712164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e19f-6cb2-41f7-858b-b7b459a756d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:47.712435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e19f-6cb2-41f7-858b-b7b459a756d1 None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:47.713154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17e19f-6cb2-41f7-858b-b7b459a756d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:47.713455 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17e19f-6cb2-41f7-858b-b7b459a756d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:47.714888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17e19f-6cb2-41f7-858b-b7b459a756d1 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:47.732347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 409/825] 10.222.0.58 () {56 vars in 1133 bytes} [Fri Oct 28 18:54:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:47.737942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5df98-803b-43f3-af53-a3029a2d67b8 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:47.833230 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:47.837210 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5df98-803b-43f3-af53-a3029a2d67b8 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:47.871048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfeb37-2e83-47a8-8ee0-fe6e5c9254a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:47.871653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfeb37-2e83-47a8-8ee0-fe6e5c9254a1 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:47.871930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfeb37-2e83-47a8-8ee0-fe6e5c9254a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:47.872198 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfeb37-2e83-47a8-8ee0-fe6e5c9254a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:47.936953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfeb37-2e83-47a8-8ee0-fe6e5c9254a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:47.938389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfeb37-2e83-47a8-8ee0-fe6e5c9254a1 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:48.029013 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:48.035151 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 417/826] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:54:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 28 18:54:48.039411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17e19f-6cb2-41f7-858b-b7b459a756d1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:48.133767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17e19f-6cb2-41f7-858b-b7b459a756d1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:48.166785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c1372-30ac-430a-b71c-6619f2dc7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:48.167418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c1372-30ac-430a-b71c-6619f2dc7713 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:48.167693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c1372-30ac-430a-b71c-6619f2dc7713 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:48.167958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c1372-30ac-430a-b71c-6619f2dc7713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:48.177420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c1372-30ac-430a-b71c-6619f2dc7713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:48.178837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c1372-30ac-430a-b71c-6619f2dc7713 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-a62c1372-30ac-430a-b71c-6619f2dc7713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:54:48.192776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:48.193878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:54:48.194781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:54:48.194781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:54:48.194781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:48.194781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 410/827] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:48.199998 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfeb37-2e83-47a8-8ee0-fe6e5c9254a1 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:48.294232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfeb37-2e83-47a8-8ee0-fe6e5c9254a1 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:48.326612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5384-3b3e-4227-aeab-028a2807d562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:48.327236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5384-3b3e-4227-aeab-028a2807d562 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:48.327515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5384-3b3e-4227-aeab-028a2807d562 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:48.327786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5384-3b3e-4227-aeab-028a2807d562 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:48.328420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5384-3b3e-4227-aeab-028a2807d562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:48.328719 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5384-3b3e-4227-aeab-028a2807d562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:48.330063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5384-3b3e-4227-aeab-028a2807d562 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:48.341571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 418/828] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:48.347178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c1372-30ac-430a-b71c-6619f2dc7713 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:48.350963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:48.448388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c1372-30ac-430a-b71c-6619f2dc7713 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:48.480940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b8593-18e3-4551-8614-dd51232c4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:48.481550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b8593-18e3-4551-8614-dd51232c4cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:48.481824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b8593-18e3-4551-8614-dd51232c4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:48.482091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b8593-18e3-4551-8614-dd51232c4cf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:48.488545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b8593-18e3-4551-8614-dd51232c4cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:48.489965 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b8593-18e3-4551-8614-dd51232c4cf7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-f15b8593-18e3-4551-8614-dd51232c4cf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:54:48.501391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:48.502546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:48.503533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:54:48.503533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:54:48.503533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:54:48.503533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:54:48.503533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:54:48.503533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:54:48.503533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:54:48.503533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:54:48.503533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 411/829] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:54:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:54:48.508238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5384-3b3e-4227-aeab-028a2807d562 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:48.564279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:48.604659 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5384-3b3e-4227-aeab-028a2807d562 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:48.636944 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacdb51-6a04-49cf-9a4b-281d1dd2b72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:48.637532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacdb51-6a04-49cf-9a4b-281d1dd2b72b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:48.637807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacdb51-6a04-49cf-9a4b-281d1dd2b72b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:48.638074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacdb51-6a04-49cf-9a4b-281d1dd2b72b None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:48.638744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacdb51-6a04-49cf-9a4b-281d1dd2b72b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:48.639071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacdb51-6a04-49cf-9a4b-281d1dd2b72b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:48.640481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacdb51-6a04-49cf-9a4b-281d1dd2b72b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:48.652506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 419/830] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:54:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:54:48.658470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b8593-18e3-4551-8614-dd51232c4cf7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:48.718366 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:48.759976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b8593-18e3-4551-8614-dd51232c4cf7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:48.795679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f8efb-c7c1-4e8d-9dea-a37242cbf873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:48.796279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f8efb-c7c1-4e8d-9dea-a37242cbf873 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:48.796556 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f8efb-c7c1-4e8d-9dea-a37242cbf873 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:48.796825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f8efb-c7c1-4e8d-9dea-a37242cbf873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:48.797538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f8efb-c7c1-4e8d-9dea-a37242cbf873 None admin] RBAC: Adding query filter params (role.id=2604cfbe6860499e9e894574ea9c0665, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=a758611cf8d94e8ca8aad6a75bef5e34) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:48.797843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f8efb-c7c1-4e8d-9dea-a37242cbf873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:48.799301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f8efb-c7c1-4e8d-9dea-a37242cbf873 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:48.815103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 412/831] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:48 2022] GET /identity/v3/role_assignments?user.id=a758611cf8d94e8ca8aad6a75bef5e34&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=2604cfbe6860499e9e894574ea9c0665 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:48.871356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67e25e-7481-4090-a1f4-f4f393b7403c None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:48.871918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67e25e-7481-4090-a1f4-f4f393b7403c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:48.871918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67e25e-7481-4090-a1f4-f4f393b7403c None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:48.873220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 420/832] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:48.879087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a53719-e66c-4e32-ba82-1ddbb255eeea None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:48.879375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a53719-e66c-4e32-ba82-1ddbb255eeea None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:48.879649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a53719-e66c-4e32-ba82-1ddbb255eeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:48.898816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-56a53719-e66c-4e32-ba82-1ddbb255eeea None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:48.900067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-56a53719-e66c-4e32-ba82-1ddbb255eeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:48.971105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 413/833] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:49.239953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67e25e-7481-4090-a1f4-f4f393b7403c None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:49.341865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67e25e-7481-4090-a1f4-f4f393b7403c None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:49.359153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:49.381880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37fd29-ef58-4a38-af58-4c646da3bd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:49.382503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37fd29-ef58-4a38-af58-4c646da3bd44 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:49.382775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37fd29-ef58-4a38-af58-4c646da3bd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:49.383077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37fd29-ef58-4a38-af58-4c646da3bd44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:49.393877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37fd29-ef58-4a38-af58-4c646da3bd44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-2d37fd29-ef58-4a38-af58-4c646da3bd44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:54:49.398626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:54:49.399654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:54:49.400486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 421/834] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:54:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:54:49.405922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a53719-e66c-4e32-ba82-1ddbb255eeea None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:49.464144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:49.506042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a53719-e66c-4e32-ba82-1ddbb255eeea None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:49.541002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ff60a-b1d1-4ae2-85a5-67fce9482b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:49.541628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ff60a-b1d1-4ae2-85a5-67fce9482b58 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:49.541906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ff60a-b1d1-4ae2-85a5-67fce9482b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:49.542168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ff60a-b1d1-4ae2-85a5-67fce9482b58 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:49.542837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ff60a-b1d1-4ae2-85a5-67fce9482b58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:49.543182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ff60a-b1d1-4ae2-85a5-67fce9482b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:49.544531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ff60a-b1d1-4ae2-85a5-67fce9482b58 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:49.552858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 414/835] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:54:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:49.559743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37fd29-ef58-4a38-af58-4c646da3bd44 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:49.660658 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37fd29-ef58-4a38-af58-4c646da3bd44 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:49.693930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8b15-ec3b-40b6-b08f-1e6ba6208a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:49.694549 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8b15-ec3b-40b6-b08f-1e6ba6208a3e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:49.694825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8b15-ec3b-40b6-b08f-1e6ba6208a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:49.695129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8b15-ec3b-40b6-b08f-1e6ba6208a3e None admin] PATH_INFO: `/v3/users/dd7eee59c750497a8f83d7c078736619` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:49.696239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b8b15-ec3b-40b6-b08f-1e6ba6208a3e None admin] RBAC: Authorizing `identity:get_user(user_id=dd7eee59c750497a8f83d7c078736619)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:49.697662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b8b15-ec3b-40b6-b08f-1e6ba6208a3e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:49.699282 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 422/836] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:54:49 2022] GET /identity/v3/users/dd7eee59c750497a8f83d7c078736619 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:49.706529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ff60a-b1d1-4ae2-85a5-67fce9482b58 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:49.802001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ff60a-b1d1-4ae2-85a5-67fce9482b58 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:49.821147 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:49.834893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e675533-f945-4f2f-896b-5f288082befb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:49.835520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e675533-f945-4f2f-896b-5f288082befb None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:49.835798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e675533-f945-4f2f-896b-5f288082befb None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:49.836066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e675533-f945-4f2f-896b-5f288082befb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:49.900336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e675533-f945-4f2f-896b-5f288082befb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:49.901805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e675533-f945-4f2f-896b-5f288082befb None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:50.000707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 415/837] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:54:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:54:50.006642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8b15-ec3b-40b6-b08f-1e6ba6208a3e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:50.051240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:50.101123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8b15-ec3b-40b6-b08f-1e6ba6208a3e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:50.132394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34922791-c398-4f86-80d0-bf2998578725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:50.132996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34922791-c398-4f86-80d0-bf2998578725 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:50.133270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34922791-c398-4f86-80d0-bf2998578725 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:50.133538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34922791-c398-4f86-80d0-bf2998578725 None admin] PATH_INFO: `/v3/projects/15e797a01bfd43da9e2a7368a3b17749` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:50.134617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34922791-c398-4f86-80d0-bf2998578725 None admin] RBAC: Authorizing `identity:get_project(project_id=15e797a01bfd43da9e2a7368a3b17749)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:50.136090 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34922791-c398-4f86-80d0-bf2998578725 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:50.137653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 423/838] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:54:50 2022] GET /identity/v3/projects/15e797a01bfd43da9e2a7368a3b17749 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:50.143823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e675533-f945-4f2f-896b-5f288082befb None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:50.164409 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:50.239563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e675533-f945-4f2f-896b-5f288082befb None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:50.271609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431bc32-5527-4fb7-aeb3-536fc8f4bfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:50.272188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431bc32-5527-4fb7-aeb3-536fc8f4bfa2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:50.272465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431bc32-5527-4fb7-aeb3-536fc8f4bfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:50.272734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431bc32-5527-4fb7-aeb3-536fc8f4bfa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:50.273448 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431bc32-5527-4fb7-aeb3-536fc8f4bfa2 None admin] RBAC: Adding query filter params (role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.project.id=15e797a01bfd43da9e2a7368a3b17749, user.id=dd7eee59c750497a8f83d7c078736619) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:50.273747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431bc32-5527-4fb7-aeb3-536fc8f4bfa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:50.275177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431bc32-5527-4fb7-aeb3-536fc8f4bfa2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:50.281196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 416/839] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:50 2022] GET /identity/v3/role_assignments?user.id=dd7eee59c750497a8f83d7c078736619&scope.project.id=15e797a01bfd43da9e2a7368a3b17749&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:52.406796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1be5c6-6548-42d8-bd58-8ee1402af2e3 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:52.407407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1be5c6-6548-42d8-bd58-8ee1402af2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:52.407407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1be5c6-6548-42d8-bd58-8ee1402af2e3 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:52.408678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 424/840] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:52.413799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422de5da-1dd8-4ef3-b0f1-cc814425b793 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:52.414189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422de5da-1dd8-4ef3-b0f1-cc814425b793 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:52.414332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422de5da-1dd8-4ef3-b0f1-cc814425b793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:52.431983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-422de5da-1dd8-4ef3-b0f1-cc814425b793 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:52.433786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-422de5da-1dd8-4ef3-b0f1-cc814425b793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:52.497697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 417/841] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:52.767672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1be5c6-6548-42d8-bd58-8ee1402af2e3 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:52.863635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1be5c6-6548-42d8-bd58-8ee1402af2e3 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:52.877437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:52.896602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366a238-907d-43c5-b556-a4aba53766d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:52.897212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366a238-907d-43c5-b556-a4aba53766d4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:52.897487 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366a238-907d-43c5-b556-a4aba53766d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:52.897754 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366a238-907d-43c5-b556-a4aba53766d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:52.898831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366a238-907d-43c5-b556-a4aba53766d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:52.900267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366a238-907d-43c5-b556-a4aba53766d4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:52.901820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 425/842] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:54:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:52.907521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422de5da-1dd8-4ef3-b0f1-cc814425b793 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:52.922517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:53.004180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422de5da-1dd8-4ef3-b0f1-cc814425b793 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:53.150419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e0ab8-dc10-4e7c-bcd7-82da10dc280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:53.151548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e0ab8-dc10-4e7c-bcd7-82da10dc280b None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:53.151823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e0ab8-dc10-4e7c-bcd7-82da10dc280b None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:53.152086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e0ab8-dc10-4e7c-bcd7-82da10dc280b None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:53.152724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e0ab8-dc10-4e7c-bcd7-82da10dc280b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:53.154113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e0ab8-dc10-4e7c-bcd7-82da10dc280b None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:53.175041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 418/843] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:54:52 2022] POST /identity/v3/users => generated 286 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:54:55.294735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59394a82-fb5f-4388-890d-c64cfafe8ec7 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:55.295321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59394a82-fb5f-4388-890d-c64cfafe8ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:55.295321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59394a82-fb5f-4388-890d-c64cfafe8ec7 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:55.296611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 426/844] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:55.301621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8960636-1933-4751-9782-9c16b526412d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:55.301998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8960636-1933-4751-9782-9c16b526412d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:55.302149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8960636-1933-4751-9782-9c16b526412d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:55.319685 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a8960636-1933-4751-9782-9c16b526412d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:55.320838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a8960636-1933-4751-9782-9c16b526412d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:55.379841 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:55.384833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 419/845] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:55.650538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59394a82-fb5f-4388-890d-c64cfafe8ec7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:55.747773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59394a82-fb5f-4388-890d-c64cfafe8ec7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:55.782016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f797492-5960-4abb-9d48-f6e291c50d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:55.782636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f797492-5960-4abb-9d48-f6e291c50d66 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:55.782908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f797492-5960-4abb-9d48-f6e291c50d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:55.783199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f797492-5960-4abb-9d48-f6e291c50d66 None admin] PATH_INFO: `/v3/roles/0481df198a1a42f0a00358e0b9a938c5` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:55.784548 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f797492-5960-4abb-9d48-f6e291c50d66 None admin] RBAC: Authorizing `identity:get_role(role_id=0481df198a1a42f0a00358e0b9a938c5)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:55.785883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f797492-5960-4abb-9d48-f6e291c50d66 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:55.787198 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 427/846] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:54:55 2022] GET /identity/v3/roles/0481df198a1a42f0a00358e0b9a938c5 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:55.793010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8960636-1933-4751-9782-9c16b526412d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:55.886827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8960636-1933-4751-9782-9c16b526412d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:55.919570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa1e49-d934-44ca-81bb-f450baeeb9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:55.920115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa1e49-d934-44ca-81bb-f450baeeb9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:55.920395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa1e49-d934-44ca-81bb-f450baeeb9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:55.920661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa1e49-d934-44ca-81bb-f450baeeb9f5 None admin] PATH_INFO: `/v3/users/1e5c3ee2c52c4bacb642ffeab15f19a7` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:55.933189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa1e49-d934-44ca-81bb-f450baeeb9f5 None admin] RBAC: Authorizing `identity:get_user(user_id=1e5c3ee2c52c4bacb642ffeab15f19a7)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:55.934607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa1e49-d934-44ca-81bb-f450baeeb9f5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:55.936192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 420/847] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:54:55 2022] GET /identity/v3/users/1e5c3ee2c52c4bacb642ffeab15f19a7 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:55.941838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f797492-5960-4abb-9d48-f6e291c50d66 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:55.989364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:56.037536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f797492-5960-4abb-9d48-f6e291c50d66 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:56.069969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edd2e9-84a4-422b-94e2-5bfb5912a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:56.070578 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edd2e9-84a4-422b-94e2-5bfb5912a902 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:56.070851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edd2e9-84a4-422b-94e2-5bfb5912a902 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:56.071146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edd2e9-84a4-422b-94e2-5bfb5912a902 None admin] PATH_INFO: `/v3/projects/15e797a01bfd43da9e2a7368a3b17749` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:56.072230 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edd2e9-84a4-422b-94e2-5bfb5912a902 None admin] RBAC: Authorizing `identity:get_project(project_id=15e797a01bfd43da9e2a7368a3b17749)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:56.073661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edd2e9-84a4-422b-94e2-5bfb5912a902 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:56.075258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 428/848] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:54:55 2022] GET /identity/v3/projects/15e797a01bfd43da9e2a7368a3b17749 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:56.081201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa1e49-d934-44ca-81bb-f450baeeb9f5 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:56.136954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:56.177810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa1e49-d934-44ca-81bb-f450baeeb9f5 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:56.212577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaecbb-9f76-42a9-8731-1b2fd653c88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:56.213744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaecbb-9f76-42a9-8731-1b2fd653c88c None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:56.214021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaecbb-9f76-42a9-8731-1b2fd653c88c None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:56.214289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaecbb-9f76-42a9-8731-1b2fd653c88c None admin] PATH_INFO: `/v3/projects/15e797a01bfd43da9e2a7368a3b17749/users/1e5c3ee2c52c4bacb642ffeab15f19a7/roles/0481df198a1a42f0a00358e0b9a938c5` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:56.216397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaecbb-9f76-42a9-8731-1b2fd653c88c None admin] RBAC: Authorizing `identity:create_grant(project_id=15e797a01bfd43da9e2a7368a3b17749, user_id=1e5c3ee2c52c4bacb642ffeab15f19a7, role_id=0481df198a1a42f0a00358e0b9a938c5)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:56.217812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaecbb-9f76-42a9-8731-1b2fd653c88c None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:56.230360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 421/849] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:54:56 2022] PUT /identity/v3/projects/15e797a01bfd43da9e2a7368a3b17749/users/1e5c3ee2c52c4bacb642ffeab15f19a7/roles/0481df198a1a42f0a00358e0b9a938c5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:54:58.321973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef22330-1ea2-4d2f-8a9e-34633c034069 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:58.322508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef22330-1ea2-4d2f-8a9e-34633c034069 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:58.322508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef22330-1ea2-4d2f-8a9e-34633c034069 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:58.323895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 429/850] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:54:58.328928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad03b6-0393-45ac-9a29-66f1d99447ae None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:58.329358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad03b6-0393-45ac-9a29-66f1d99447ae None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:58.329463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad03b6-0393-45ac-9a29-66f1d99447ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:58.347596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-baad03b6-0393-45ac-9a29-66f1d99447ae None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:54:58.348765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-baad03b6-0393-45ac-9a29-66f1d99447ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:54:58.412372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 422/851] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:54:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:54:58.678263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef22330-1ea2-4d2f-8a9e-34633c034069 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:58.776244 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef22330-1ea2-4d2f-8a9e-34633c034069 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:58.792427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:58.808465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe998e72-1946-4b70-96e3-d00213d3104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:58.809081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe998e72-1946-4b70-96e3-d00213d3104c None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:58.809357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe998e72-1946-4b70-96e3-d00213d3104c None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:58.809626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe998e72-1946-4b70-96e3-d00213d3104c None admin] PATH_INFO: `/v3/roles/0481df198a1a42f0a00358e0b9a938c5` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:58.811009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe998e72-1946-4b70-96e3-d00213d3104c None admin] RBAC: Authorizing `identity:get_role(role_id=0481df198a1a42f0a00358e0b9a938c5)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:58.812366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe998e72-1946-4b70-96e3-d00213d3104c None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:58.813656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 430/852] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:54:58 2022] GET /identity/v3/roles/0481df198a1a42f0a00358e0b9a938c5 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:58.819365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad03b6-0393-45ac-9a29-66f1d99447ae None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:58.865848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:58.915844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad03b6-0393-45ac-9a29-66f1d99447ae None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:58.947956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58655f5b-d91a-4a80-9fc5-9da83753265a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:58.948538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58655f5b-d91a-4a80-9fc5-9da83753265a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:58.948816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58655f5b-d91a-4a80-9fc5-9da83753265a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:58.949084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58655f5b-d91a-4a80-9fc5-9da83753265a None admin] PATH_INFO: `/v3/users/1e5c3ee2c52c4bacb642ffeab15f19a7` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:58.950222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58655f5b-d91a-4a80-9fc5-9da83753265a None admin] RBAC: Authorizing `identity:get_user(user_id=1e5c3ee2c52c4bacb642ffeab15f19a7)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:58.951673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58655f5b-d91a-4a80-9fc5-9da83753265a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:58.953218 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 423/853] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:54:58 2022] GET /identity/v3/users/1e5c3ee2c52c4bacb642ffeab15f19a7 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:58.958864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe998e72-1946-4b70-96e3-d00213d3104c None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:59.053404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe998e72-1946-4b70-96e3-d00213d3104c None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:59.086175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1790cba-33e4-4127-9b42-b8d0134f17eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:59.086765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1790cba-33e4-4127-9b42-b8d0134f17eb None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:59.087074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1790cba-33e4-4127-9b42-b8d0134f17eb None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:59.087348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1790cba-33e4-4127-9b42-b8d0134f17eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:59.138525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:59.152974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1790cba-33e4-4127-9b42-b8d0134f17eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:59.154386 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1790cba-33e4-4127-9b42-b8d0134f17eb None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:59.246959 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 431/854] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:54:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:54:59.252754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58655f5b-d91a-4a80-9fc5-9da83753265a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:59.348494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58655f5b-d91a-4a80-9fc5-9da83753265a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:59.365079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:59.382689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164df2a-2722-4487-bfc8-f3efd5768b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:59.383341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164df2a-2722-4487-bfc8-f3efd5768b4d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:59.383616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164df2a-2722-4487-bfc8-f3efd5768b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:59.383884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164df2a-2722-4487-bfc8-f3efd5768b4d None admin] PATH_INFO: `/v3/projects/15e797a01bfd43da9e2a7368a3b17749` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:59.384999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164df2a-2722-4487-bfc8-f3efd5768b4d None admin] RBAC: Authorizing `identity:get_project(project_id=15e797a01bfd43da9e2a7368a3b17749)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:59.386436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164df2a-2722-4487-bfc8-f3efd5768b4d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:59.388050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 424/855] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:54:59 2022] GET /identity/v3/projects/15e797a01bfd43da9e2a7368a3b17749 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:54:59.394612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1790cba-33e4-4127-9b42-b8d0134f17eb None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:54:59.473939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:54:59.488709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1790cba-33e4-4127-9b42-b8d0134f17eb None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:54:59.521771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94b12e-62df-4fd8-aeaa-b17a8de2e7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:54:59.522361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94b12e-62df-4fd8-aeaa-b17a8de2e7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:54:59.522637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94b12e-62df-4fd8-aeaa-b17a8de2e7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:54:59.522904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94b12e-62df-4fd8-aeaa-b17a8de2e7f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:54:59.523643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94b12e-62df-4fd8-aeaa-b17a8de2e7f7 None admin] RBAC: Adding query filter params (role.id=0481df198a1a42f0a00358e0b9a938c5, scope.project.id=15e797a01bfd43da9e2a7368a3b17749, user.id=1e5c3ee2c52c4bacb642ffeab15f19a7) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:54:59.523946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94b12e-62df-4fd8-aeaa-b17a8de2e7f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:54:59.525356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94b12e-62df-4fd8-aeaa-b17a8de2e7f7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:54:59.530861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 432/856] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:54:59 2022] GET /identity/v3/role_assignments?user.id=1e5c3ee2c52c4bacb642ffeab15f19a7&scope.project.id=15e797a01bfd43da9e2a7368a3b17749&role.id=0481df198a1a42f0a00358e0b9a938c5 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:01.635925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13febe91-4b26-48e2-9d30-39e34bcdb9ef None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:01.636459 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13febe91-4b26-48e2-9d30-39e34bcdb9ef None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:01.636459 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13febe91-4b26-48e2-9d30-39e34bcdb9ef None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:01.637786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 425/857] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:55:01.642814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b6fb-f6e5-4692-8016-cde1dac6c02e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:01.643234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b6fb-f6e5-4692-8016-cde1dac6c02e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:01.643653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b6fb-f6e5-4692-8016-cde1dac6c02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:01.661711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-51e8b6fb-f6e5-4692-8016-cde1dac6c02e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:55:01.662867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-51e8b6fb-f6e5-4692-8016-cde1dac6c02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:55:01.725039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 433/858] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:55:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:55:01.992292 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13febe91-4b26-48e2-9d30-39e34bcdb9ef None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:02.090228 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13febe91-4b26-48e2-9d30-39e34bcdb9ef None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:02.122303 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebfdba-5364-45df-8ae7-b04480db0a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:02.122906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebfdba-5364-45df-8ae7-b04480db0a32 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:02.123204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebfdba-5364-45df-8ae7-b04480db0a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:02.123468 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebfdba-5364-45df-8ae7-b04480db0a32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:02.124551 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebfdba-5364-45df-8ae7-b04480db0a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:02.125945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebfdba-5364-45df-8ae7-b04480db0a32 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:02.127492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 426/859] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:55:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:02.133365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b6fb-f6e5-4692-8016-cde1dac6c02e None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:02.209224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:02.227840 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b6fb-f6e5-4692-8016-cde1dac6c02e None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:02.260648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d645618-232d-4222-85dc-ca082d4b58a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:02.261238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d645618-232d-4222-85dc-ca082d4b58a0 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:02.261510 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d645618-232d-4222-85dc-ca082d4b58a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:02.261773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d645618-232d-4222-85dc-ca082d4b58a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:02.262421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d645618-232d-4222-85dc-ca082d4b58a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:02.263855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d645618-232d-4222-85dc-ca082d4b58a0 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:02.288411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 434/860] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:55:02 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:55:04.442013 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8767e-1bdd-4e84-93ca-42976fbbac0f None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:04.442545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8767e-1bdd-4e84-93ca-42976fbbac0f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:04.442545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8767e-1bdd-4e84-93ca-42976fbbac0f None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:04.443876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 427/861] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:55:04.449073 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776e581-6644-4ebf-aae6-29dd21df589b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:04.449506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776e581-6644-4ebf-aae6-29dd21df589b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:04.449600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776e581-6644-4ebf-aae6-29dd21df589b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:04.467950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-7776e581-6644-4ebf-aae6-29dd21df589b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:55:04.469119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7776e581-6644-4ebf-aae6-29dd21df589b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:55:04.532189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 435/862] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:55:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:55:04.801112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8767e-1bdd-4e84-93ca-42976fbbac0f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:04.852978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:04.899269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8767e-1bdd-4e84-93ca-42976fbbac0f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:04.931261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acbbd7-c4d5-4ca6-bbd1-97be3375c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:04.931866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acbbd7-c4d5-4ca6-bbd1-97be3375c412 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:04.932209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acbbd7-c4d5-4ca6-bbd1-97be3375c412 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:04.932480 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acbbd7-c4d5-4ca6-bbd1-97be3375c412 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:04.933573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acbbd7-c4d5-4ca6-bbd1-97be3375c412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:04.935016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acbbd7-c4d5-4ca6-bbd1-97be3375c412 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:04.936550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 428/863] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:55:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:04.942134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776e581-6644-4ebf-aae6-29dd21df589b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:05.000681 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:05.038885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776e581-6644-4ebf-aae6-29dd21df589b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:05.071587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb640f9-e909-4d3d-8039-b86a2e6817af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:05.072212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb640f9-e909-4d3d-8039-b86a2e6817af None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:05.073062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb640f9-e909-4d3d-8039-b86a2e6817af None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:05.073329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb640f9-e909-4d3d-8039-b86a2e6817af None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:05.073974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb640f9-e909-4d3d-8039-b86a2e6817af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:05.075409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb640f9-e909-4d3d-8039-b86a2e6817af None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:05.095243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 436/864] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:55:04 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:55:07.284021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2db9b2-44c3-49e8-a8ea-63c389f31204 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:07.284593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2db9b2-44c3-49e8-a8ea-63c389f31204 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:07.284593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2db9b2-44c3-49e8-a8ea-63c389f31204 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:07.285856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 429/865] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:55:07.291073 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308268e6-d419-4714-96c4-c7d08ad94777 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:07.291445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308268e6-d419-4714-96c4-c7d08ad94777 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:07.291615 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308268e6-d419-4714-96c4-c7d08ad94777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:07.309850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-308268e6-d419-4714-96c4-c7d08ad94777 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:55:07.311040 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-308268e6-d419-4714-96c4-c7d08ad94777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:55:07.374698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 437/866] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:55:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:55:07.636895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2db9b2-44c3-49e8-a8ea-63c389f31204 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:07.734452 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2db9b2-44c3-49e8-a8ea-63c389f31204 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:07.754306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:07.769079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd1782-d792-4a7d-b0bd-c64c274d9eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:07.769700 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd1782-d792-4a7d-b0bd-c64c274d9eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:07.769974 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd1782-d792-4a7d-b0bd-c64c274d9eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:07.770242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd1782-d792-4a7d-b0bd-c64c274d9eb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:07.780771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd1782-d792-4a7d-b0bd-c64c274d9eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-d4cd1782-d792-4a7d-b0bd-c64c274d9eb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:55:07.785564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:55:07.786642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:55:07.787880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 430/867] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:55:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:55:07.792638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308268e6-d419-4714-96c4-c7d08ad94777 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:07.838607 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:07.887338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308268e6-d419-4714-96c4-c7d08ad94777 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:07.919521 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931b544-c97a-4c81-abb6-b735034ea4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:07.920106 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931b544-c97a-4c81-abb6-b735034ea4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:07.920380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931b544-c97a-4c81-abb6-b735034ea4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:07.920645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931b544-c97a-4c81-abb6-b735034ea4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:07.921289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931b544-c97a-4c81-abb6-b735034ea4a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:55:07.921586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931b544-c97a-4c81-abb6-b735034ea4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:07.922926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931b544-c97a-4c81-abb6-b735034ea4a6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:07.930496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 438/868] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:55:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:07.936679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd1782-d792-4a7d-b0bd-c64c274d9eb7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:08.031750 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd1782-d792-4a7d-b0bd-c64c274d9eb7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:08.063794 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44328f-c9c3-4e4b-8b57-f554c16d1cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:08.064392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44328f-c9c3-4e4b-8b57-f554c16d1cdb None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:08.064669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44328f-c9c3-4e4b-8b57-f554c16d1cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:08.064939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44328f-c9c3-4e4b-8b57-f554c16d1cdb None admin] PATH_INFO: `/v3/users/7f7517721eb34b4092ab55cc7b26e4e5` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:08.077747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44328f-c9c3-4e4b-8b57-f554c16d1cdb None admin] RBAC: Authorizing `identity:get_user(user_id=7f7517721eb34b4092ab55cc7b26e4e5)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:08.079197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44328f-c9c3-4e4b-8b57-f554c16d1cdb None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:08.080736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 431/869] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:55:07 2022] GET /identity/v3/users/7f7517721eb34b4092ab55cc7b26e4e5 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:08.086224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931b544-c97a-4c81-abb6-b735034ea4a6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:08.187274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931b544-c97a-4c81-abb6-b735034ea4a6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:08.196809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:08.219747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98857a6-8d88-4b0f-ab72-6924951fb267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:08.220360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98857a6-8d88-4b0f-ab72-6924951fb267 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:08.220634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98857a6-8d88-4b0f-ab72-6924951fb267 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:08.220902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98857a6-8d88-4b0f-ab72-6924951fb267 None admin] PATH_INFO: `/v3/projects/960c6e2629a2472dbbf39558ba266161` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:08.221992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98857a6-8d88-4b0f-ab72-6924951fb267 None admin] RBAC: Authorizing `identity:get_project(project_id=960c6e2629a2472dbbf39558ba266161)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:08.223438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98857a6-8d88-4b0f-ab72-6924951fb267 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:08.225024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 439/870] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:55:08 2022] GET /identity/v3/projects/960c6e2629a2472dbbf39558ba266161 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:08.230804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44328f-c9c3-4e4b-8b57-f554c16d1cdb None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:08.270248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:08.328073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44328f-c9c3-4e4b-8b57-f554c16d1cdb None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:08.360652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9ab7b-6c66-469d-8de5-9c9dada98db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:08.361290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9ab7b-6c66-469d-8de5-9c9dada98db5 None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:08.361569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9ab7b-6c66-469d-8de5-9c9dada98db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:08.361843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9ab7b-6c66-469d-8de5-9c9dada98db5 None admin] PATH_INFO: `/v3/projects/960c6e2629a2472dbbf39558ba266161/users/7f7517721eb34b4092ab55cc7b26e4e5/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:08.363722 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9ab7b-6c66-469d-8de5-9c9dada98db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=960c6e2629a2472dbbf39558ba266161, user_id=7f7517721eb34b4092ab55cc7b26e4e5, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:08.365139 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9ab7b-6c66-469d-8de5-9c9dada98db5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:08.378012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 432/871] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:55:08 2022] PUT /identity/v3/projects/960c6e2629a2472dbbf39558ba266161/users/7f7517721eb34b4092ab55cc7b26e4e5/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:55:10.480112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbc4bf-3bc7-41ef-b135-45fb4ed472e6 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:10.480644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbc4bf-3bc7-41ef-b135-45fb4ed472e6 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:10.480773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbc4bf-3bc7-41ef-b135-45fb4ed472e6 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:10.482010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 440/872] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:55:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:55:10.487266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a164262-a7bb-466a-b972-171127c066c2 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:10.487693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a164262-a7bb-466a-b972-171127c066c2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:10.487793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a164262-a7bb-466a-b972-171127c066c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:10.505214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-9a164262-a7bb-466a-b972-171127c066c2 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:55:10.506368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9a164262-a7bb-466a-b972-171127c066c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:55:10.569694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 433/873] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:55:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:55:10.839344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbc4bf-3bc7-41ef-b135-45fb4ed472e6 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:10.936394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbc4bf-3bc7-41ef-b135-45fb4ed472e6 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:10.968922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebba3d5-4875-4f68-936b-cdcb84d2f0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:10.969529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebba3d5-4875-4f68-936b-cdcb84d2f0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:10.969802 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebba3d5-4875-4f68-936b-cdcb84d2f0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:10.970066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebba3d5-4875-4f68-936b-cdcb84d2f0e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:10.979671 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebba3d5-4875-4f68-936b-cdcb84d2f0e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-eebba3d5-4875-4f68-936b-cdcb84d2f0e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:55:10.984465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:55:10.985530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:55:10.986490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 441/874] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:55:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:55:10.991288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a164262-a7bb-466a-b972-171127c066c2 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:11.018783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:11.087061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a164262-a7bb-466a-b972-171127c066c2 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:11.118453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc188a-57df-446e-8ed7-6ad5da870002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:11.119577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc188a-57df-446e-8ed7-6ad5da870002 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:11.119852 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc188a-57df-446e-8ed7-6ad5da870002 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:11.120115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc188a-57df-446e-8ed7-6ad5da870002 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:11.120756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc188a-57df-446e-8ed7-6ad5da870002 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:55:11.121049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc188a-57df-446e-8ed7-6ad5da870002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:11.122392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc188a-57df-446e-8ed7-6ad5da870002 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:11.130050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 434/875] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:55:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:11.136488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebba3d5-4875-4f68-936b-cdcb84d2f0e3 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:11.174765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:11.232528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebba3d5-4875-4f68-936b-cdcb84d2f0e3 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:11.265373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c29028-1e82-4cf9-88a9-075a3c920418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:11.265986 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c29028-1e82-4cf9-88a9-075a3c920418 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:11.266264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c29028-1e82-4cf9-88a9-075a3c920418 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:11.266531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c29028-1e82-4cf9-88a9-075a3c920418 None admin] PATH_INFO: `/v3/users/7f7517721eb34b4092ab55cc7b26e4e5` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:11.267614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c29028-1e82-4cf9-88a9-075a3c920418 None admin] RBAC: Authorizing `identity:get_user(user_id=7f7517721eb34b4092ab55cc7b26e4e5)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:11.269332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c29028-1e82-4cf9-88a9-075a3c920418 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:11.270830 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 442/876] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:55:11 2022] GET /identity/v3/users/7f7517721eb34b4092ab55cc7b26e4e5 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:11.276477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc188a-57df-446e-8ed7-6ad5da870002 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:11.362427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:11.372927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc188a-57df-446e-8ed7-6ad5da870002 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:11.405394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195d8fd-951e-4b26-9507-309e2ae6a15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:11.405982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195d8fd-951e-4b26-9507-309e2ae6a15a None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:11.406259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195d8fd-951e-4b26-9507-309e2ae6a15a None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:11.406527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195d8fd-951e-4b26-9507-309e2ae6a15a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:11.470550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195d8fd-951e-4b26-9507-309e2ae6a15a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:11.472001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195d8fd-951e-4b26-9507-309e2ae6a15a None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:11.556057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:11.566441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 435/877] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:55:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:55:11.572714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c29028-1e82-4cf9-88a9-075a3c920418 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:11.668052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c29028-1e82-4cf9-88a9-075a3c920418 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:11.674630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:11.701278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be8aa8-dbed-4f2c-a9c7-534d238ac17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:11.701891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be8aa8-dbed-4f2c-a9c7-534d238ac17b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:11.702164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be8aa8-dbed-4f2c-a9c7-534d238ac17b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:11.702430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be8aa8-dbed-4f2c-a9c7-534d238ac17b None admin] PATH_INFO: `/v3/projects/960c6e2629a2472dbbf39558ba266161` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:11.703528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be8aa8-dbed-4f2c-a9c7-534d238ac17b None admin] RBAC: Authorizing `identity:get_project(project_id=960c6e2629a2472dbbf39558ba266161)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:11.704929 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be8aa8-dbed-4f2c-a9c7-534d238ac17b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:11.706512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 443/878] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:55:11 2022] GET /identity/v3/projects/960c6e2629a2472dbbf39558ba266161 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:11.713197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195d8fd-951e-4b26-9507-309e2ae6a15a None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:11.808977 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195d8fd-951e-4b26-9507-309e2ae6a15a None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:11.842207 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fa888-15ca-4bb5-974d-1fbafd037c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:11.842791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fa888-15ca-4bb5-974d-1fbafd037c72 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:11.843097 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fa888-15ca-4bb5-974d-1fbafd037c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:11.843370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fa888-15ca-4bb5-974d-1fbafd037c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:11.844081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fa888-15ca-4bb5-974d-1fbafd037c72 None admin] RBAC: Adding query filter params (role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.project.id=960c6e2629a2472dbbf39558ba266161, user.id=7f7517721eb34b4092ab55cc7b26e4e5) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:55:11.844384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fa888-15ca-4bb5-974d-1fbafd037c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:11.845799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fa888-15ca-4bb5-974d-1fbafd037c72 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:11.851303 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 436/879] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:55:11 2022] GET /identity/v3/role_assignments?user.id=7f7517721eb34b4092ab55cc7b26e4e5&scope.project.id=960c6e2629a2472dbbf39558ba266161&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:13.986853 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a78568-7e85-4c4b-8f73-accc6c5cd6ad None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:13.987546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a78568-7e85-4c4b-8f73-accc6c5cd6ad None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:13.987546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a78568-7e85-4c4b-8f73-accc6c5cd6ad None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:13.988759 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 444/880] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:55:13.994578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61a86d-414c-44f9-8774-5f936aca6675 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:13.995028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61a86d-414c-44f9-8774-5f936aca6675 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:13.995144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61a86d-414c-44f9-8774-5f936aca6675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:14.012724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-9e61a86d-414c-44f9-8774-5f936aca6675 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:55:14.013903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9e61a86d-414c-44f9-8774-5f936aca6675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:55:14.061320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:14.078061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 437/881] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:55:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:55:14.348616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a78568-7e85-4c4b-8f73-accc6c5cd6ad None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:14.446723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a78568-7e85-4c4b-8f73-accc6c5cd6ad None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:14.480452 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c349b-3b1f-484d-b646-60908a594fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:14.481043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c349b-3b1f-484d-b646-60908a594fa3 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:14.481313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c349b-3b1f-484d-b646-60908a594fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:14.481577 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c349b-3b1f-484d-b646-60908a594fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:14.482225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c349b-3b1f-484d-b646-60908a594fa3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:14.483576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c349b-3b1f-484d-b646-60908a594fa3 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:14.498506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 445/882] 10.222.0.58 () {60 vars in 1138 bytes} [Fri Oct 28 18:55:14 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:55:16.647537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf625-ad6d-44bf-a695-6d2de1dc6410 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:16.648102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf625-ad6d-44bf-a695-6d2de1dc6410 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:16.648102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf625-ad6d-44bf-a695-6d2de1dc6410 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:16.649414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 438/883] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:55:16.654714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1860cbb-40e9-47c3-bf72-889276a33085 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:16.655160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1860cbb-40e9-47c3-bf72-889276a33085 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:16.655297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1860cbb-40e9-47c3-bf72-889276a33085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:16.673628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e1860cbb-40e9-47c3-bf72-889276a33085 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:55:16.674806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e1860cbb-40e9-47c3-bf72-889276a33085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:55:16.700703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:16.740151 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 446/884] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:55:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:55:17.008541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf625-ad6d-44bf-a695-6d2de1dc6410 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:17.107062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf625-ad6d-44bf-a695-6d2de1dc6410 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:17.138999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408348f-bba5-4729-a260-a9efbb75b6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:17.139605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408348f-bba5-4729-a260-a9efbb75b6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:17.139885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408348f-bba5-4729-a260-a9efbb75b6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:17.140156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408348f-bba5-4729-a260-a9efbb75b6e0 None admin] PATH_INFO: `/v3/domains/a54c6b89f46343458d2f2d6a77159b69` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:17.150874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408348f-bba5-4729-a260-a9efbb75b6e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a54c6b89f46343458d2f2d6a77159b69)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:17.152327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408348f-bba5-4729-a260-a9efbb75b6e0 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:17.153866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 439/885] 10.222.0.58 () {56 vars in 1183 bytes} [Fri Oct 28 18:55:17 2022] GET /identity/v3/domains/a54c6b89f46343458d2f2d6a77159b69 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:17.159560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1860cbb-40e9-47c3-bf72-889276a33085 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:17.252650 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1860cbb-40e9-47c3-bf72-889276a33085 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:17.284891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc65a9-ea50-4be5-898e-415ca5611cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:17.285470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc65a9-ea50-4be5-898e-415ca5611cb7 None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:17.285741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc65a9-ea50-4be5-898e-415ca5611cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:17.286006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc65a9-ea50-4be5-898e-415ca5611cb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:17.286639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc65a9-ea50-4be5-898e-415ca5611cb7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:17.288070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc65a9-ea50-4be5-898e-415ca5611cb7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:17.313836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 447/886] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:55:17 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:55:19.458268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2c0e-4b2c-4851-9fad-555064993529 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:19.458924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2c0e-4b2c-4851-9fad-555064993529 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:19.458924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2c0e-4b2c-4851-9fad-555064993529 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:19.460181 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 440/887] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:55:19.465807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a300947-1d8e-4c0a-ade6-d06ab6f4cbb3 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:19.466260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a300947-1d8e-4c0a-ade6-d06ab6f4cbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:19.466359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a300947-1d8e-4c0a-ade6-d06ab6f4cbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:19.484582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6a300947-1d8e-4c0a-ade6-d06ab6f4cbb3 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:55:19.485734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6a300947-1d8e-4c0a-ade6-d06ab6f4cbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:55:19.495948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:19.551247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 448/888] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:55:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:55:19.820102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e2c0e-4b2c-4851-9fad-555064993529 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:19.866052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:19.919526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e2c0e-4b2c-4851-9fad-555064993529 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:19.951725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbfd65-ae98-4d52-958a-5b5e7f0557b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:19.952330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbfd65-ae98-4d52-958a-5b5e7f0557b7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:19.952604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbfd65-ae98-4d52-958a-5b5e7f0557b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:19.952869 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbfd65-ae98-4d52-958a-5b5e7f0557b7 None admin] PATH_INFO: `/v3/domains/a54c6b89f46343458d2f2d6a77159b69` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:19.953958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbfd65-ae98-4d52-958a-5b5e7f0557b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a54c6b89f46343458d2f2d6a77159b69)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:19.955403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbfd65-ae98-4d52-958a-5b5e7f0557b7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:19.956926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 441/889] 10.222.0.58 () {56 vars in 1183 bytes} [Fri Oct 28 18:55:19 2022] GET /identity/v3/domains/a54c6b89f46343458d2f2d6a77159b69 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:19.962603 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a300947-1d8e-4c0a-ade6-d06ab6f4cbb3 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:20.058179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a300947-1d8e-4c0a-ade6-d06ab6f4cbb3 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:20.090699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb75188-b1a4-4a9b-8450-239fb6200aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:20.091314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb75188-b1a4-4a9b-8450-239fb6200aba None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:20.091593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb75188-b1a4-4a9b-8450-239fb6200aba None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:20.091866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb75188-b1a4-4a9b-8450-239fb6200aba None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:20.092503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb75188-b1a4-4a9b-8450-239fb6200aba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:20.093912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb75188-b1a4-4a9b-8450-239fb6200aba None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:20.114573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 449/890] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:55:19 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:55:22.269850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a7f84-8e7d-4c87-a5d8-75de2ece248d None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:22.270436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a7f84-8e7d-4c87-a5d8-75de2ece248d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:22.270436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a7f84-8e7d-4c87-a5d8-75de2ece248d None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:22.271739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 442/891] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:55:22.277455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c014db-1d81-4dcc-8f0a-b26e91b78b0b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:22.277878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c014db-1d81-4dcc-8f0a-b26e91b78b0b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:22.277985 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c014db-1d81-4dcc-8f0a-b26e91b78b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:22.284288 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:22.297859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a4c014db-1d81-4dcc-8f0a-b26e91b78b0b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:55:22.299071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a4c014db-1d81-4dcc-8f0a-b26e91b78b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:55:22.362804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 450/892] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:55:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:55:22.626271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a7f84-8e7d-4c87-a5d8-75de2ece248d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:22.723645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a7f84-8e7d-4c87-a5d8-75de2ece248d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:22.736025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:22.756627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c70ae-8e47-4ae1-ac8f-c12d29f855de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:22.757238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c70ae-8e47-4ae1-ac8f-c12d29f855de None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:22.757510 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c70ae-8e47-4ae1-ac8f-c12d29f855de None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:22.757775 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c70ae-8e47-4ae1-ac8f-c12d29f855de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:22.768730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c70ae-8e47-4ae1-ac8f-c12d29f855de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-0a7c70ae-8e47-4ae1-ac8f-c12d29f855de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:55:22.773417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:55:22.774590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:55:22.776318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 443/893] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:55:22.781032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c014db-1d81-4dcc-8f0a-b26e91b78b0b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:22.874400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c014db-1d81-4dcc-8f0a-b26e91b78b0b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:22.904005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:22.908938 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1a3cc-fd6c-456a-b0d9-a0a14de3044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:22.909523 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1a3cc-fd6c-456a-b0d9-a0a14de3044e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:22.909797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1a3cc-fd6c-456a-b0d9-a0a14de3044e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:22.910063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1a3cc-fd6c-456a-b0d9-a0a14de3044e None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:22.910707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1a3cc-fd6c-456a-b0d9-a0a14de3044e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:55:22.911027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1a3cc-fd6c-456a-b0d9-a0a14de3044e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:22.912364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1a3cc-fd6c-456a-b0d9-a0a14de3044e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:22.920156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 451/894] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:55:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:22.925891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c70ae-8e47-4ae1-ac8f-c12d29f855de None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:23.021378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c70ae-8e47-4ae1-ac8f-c12d29f855de None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:23.052654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e1b8c-b657-4ea6-9eb5-334c61e0ddf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:23.053259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e1b8c-b657-4ea6-9eb5-334c61e0ddf2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:23.053533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e1b8c-b657-4ea6-9eb5-334c61e0ddf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:23.053799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e1b8c-b657-4ea6-9eb5-334c61e0ddf2 None admin] PATH_INFO: `/v3/users/33e65685d2e940a19d61603008366726` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:23.067477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e1b8c-b657-4ea6-9eb5-334c61e0ddf2 None admin] RBAC: Authorizing `identity:get_user(user_id=33e65685d2e940a19d61603008366726)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:23.068888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e1b8c-b657-4ea6-9eb5-334c61e0ddf2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:23.070443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 444/895] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:55:22 2022] GET /identity/v3/users/33e65685d2e940a19d61603008366726 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:23.076644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1a3cc-fd6c-456a-b0d9-a0a14de3044e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:23.170675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1a3cc-fd6c-456a-b0d9-a0a14de3044e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:23.202866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fca8b5-272c-4a00-a43e-811d56a4ec6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:23.203493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fca8b5-272c-4a00-a43e-811d56a4ec6b None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:23.203767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fca8b5-272c-4a00-a43e-811d56a4ec6b None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:23.204031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fca8b5-272c-4a00-a43e-811d56a4ec6b None admin] PATH_INFO: `/v3/projects/370228fbd935417b92a050cb09cc48de` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:23.205111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fca8b5-272c-4a00-a43e-811d56a4ec6b None admin] RBAC: Authorizing `identity:get_project(project_id=370228fbd935417b92a050cb09cc48de)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:23.206511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fca8b5-272c-4a00-a43e-811d56a4ec6b None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:23.208113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 452/896] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:55:23 2022] GET /identity/v3/projects/370228fbd935417b92a050cb09cc48de => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:55:23.213671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e1b8c-b657-4ea6-9eb5-334c61e0ddf2 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:23.242213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:23.309399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e1b8c-b657-4ea6-9eb5-334c61e0ddf2 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:23.342356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a1d1c-1e4b-415f-9800-05d3cdbebfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:23.343014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a1d1c-1e4b-415f-9800-05d3cdbebfb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:23.343296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a1d1c-1e4b-415f-9800-05d3cdbebfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:23.343563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a1d1c-1e4b-415f-9800-05d3cdbebfb2 None admin] PATH_INFO: `/v3/projects/370228fbd935417b92a050cb09cc48de/users/33e65685d2e940a19d61603008366726/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:23.345415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a1d1c-1e4b-415f-9800-05d3cdbebfb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=370228fbd935417b92a050cb09cc48de, user_id=33e65685d2e940a19d61603008366726, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:23.346833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a1d1c-1e4b-415f-9800-05d3cdbebfb2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:23.360109 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 445/897] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:55:23 2022] PUT /identity/v3/projects/370228fbd935417b92a050cb09cc48de/users/33e65685d2e940a19d61603008366726/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:55:25.474943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656e75-0ad4-408b-aa0d-0328575ef0df None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:25.479626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656e75-0ad4-408b-aa0d-0328575ef0df None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:25.479626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656e75-0ad4-408b-aa0d-0328575ef0df None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:25.479626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 453/898] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:55:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:55:25.483028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7ead4-63e9-45d1-9ba8-f90902c79849 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:25.483475 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7ead4-63e9-45d1-9ba8-f90902c79849 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:25.483575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7ead4-63e9-45d1-9ba8-f90902c79849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:25.501558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-30b7ead4-63e9-45d1-9ba8-f90902c79849 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:55:25.502711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-30b7ead4-63e9-45d1-9ba8-f90902c79849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:55:25.565707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 446/899] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:55:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:55:25.838899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656e75-0ad4-408b-aa0d-0328575ef0df None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:25.898836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:25.937257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656e75-0ad4-408b-aa0d-0328575ef0df None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:25.968777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1b86c-ebab-41f2-9ee0-1ce7563b1c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:25.969385 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1b86c-ebab-41f2-9ee0-1ce7563b1c27 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:25.969660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1b86c-ebab-41f2-9ee0-1ce7563b1c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:25.969926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1b86c-ebab-41f2-9ee0-1ce7563b1c27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:25.980023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1b86c-ebab-41f2-9ee0-1ce7563b1c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-1fb1b86c-ebab-41f2-9ee0-1ce7563b1c27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:55:25.984771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:55:25.985880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:55:25.986758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 454/900] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:55:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:55:25.991690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7ead4-63e9-45d1-9ba8-f90902c79849 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:26.012621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:26.087779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7ead4-63e9-45d1-9ba8-f90902c79849 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:26.119955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052027af-5c15-4421-b781-0ba9dfd0a784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:26.120533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052027af-5c15-4421-b781-0ba9dfd0a784 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:26.120803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052027af-5c15-4421-b781-0ba9dfd0a784 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:26.121066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052027af-5c15-4421-b781-0ba9dfd0a784 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:26.121712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052027af-5c15-4421-b781-0ba9dfd0a784 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:55:26.122005 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052027af-5c15-4421-b781-0ba9dfd0a784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:26.123377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052027af-5c15-4421-b781-0ba9dfd0a784 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:26.131001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 447/901] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:55:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:26.136848 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1b86c-ebab-41f2-9ee0-1ce7563b1c27 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:26.238720 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1b86c-ebab-41f2-9ee0-1ce7563b1c27 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:26.260093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:26.271967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457df5af-3087-4a50-a792-f4a0917c4a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:26.272526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457df5af-3087-4a50-a792-f4a0917c4a6e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:26.272803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457df5af-3087-4a50-a792-f4a0917c4a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:26.273071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457df5af-3087-4a50-a792-f4a0917c4a6e None admin] PATH_INFO: `/v3/users/33e65685d2e940a19d61603008366726` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:26.274177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457df5af-3087-4a50-a792-f4a0917c4a6e None admin] RBAC: Authorizing `identity:get_user(user_id=33e65685d2e940a19d61603008366726)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:26.276159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457df5af-3087-4a50-a792-f4a0917c4a6e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:26.277710 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 455/902] 10.222.0.58 () {56 vars in 1177 bytes} [Fri Oct 28 18:55:26 2022] GET /identity/v3/users/33e65685d2e940a19d61603008366726 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:26.283419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052027af-5c15-4421-b781-0ba9dfd0a784 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:26.362676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:26.380565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052027af-5c15-4421-b781-0ba9dfd0a784 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:26.413672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2df12-d49f-4f12-b3e2-ee570cb3cbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:26.414260 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2df12-d49f-4f12-b3e2-ee570cb3cbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:26.414535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2df12-d49f-4f12-b3e2-ee570cb3cbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:26.414802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2df12-d49f-4f12-b3e2-ee570cb3cbd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:26.478920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2df12-d49f-4f12-b3e2-ee570cb3cbd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:26.480369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2df12-d49f-4f12-b3e2-ee570cb3cbd4 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:26.555769 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:26.574137 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 448/903] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:55:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:55:26.580033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457df5af-3087-4a50-a792-f4a0917c4a6e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:26.675524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457df5af-3087-4a50-a792-f4a0917c4a6e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:26.706725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273cb1a-4b61-40a6-9b19-e556a6ee56bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:26.707369 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273cb1a-4b61-40a6-9b19-e556a6ee56bd None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:26.707662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273cb1a-4b61-40a6-9b19-e556a6ee56bd None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:26.707912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273cb1a-4b61-40a6-9b19-e556a6ee56bd None admin] PATH_INFO: `/v3/projects/370228fbd935417b92a050cb09cc48de` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:26.708992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273cb1a-4b61-40a6-9b19-e556a6ee56bd None admin] RBAC: Authorizing `identity:get_project(project_id=370228fbd935417b92a050cb09cc48de)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:26.710388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273cb1a-4b61-40a6-9b19-e556a6ee56bd None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:26.712539 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 456/904] 10.222.0.58 () {56 vars in 1186 bytes} [Fri Oct 28 18:55:26 2022] GET /identity/v3/projects/370228fbd935417b92a050cb09cc48de => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:26.718357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2df12-d49f-4f12-b3e2-ee570cb3cbd4 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:55:26.813470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2df12-d49f-4f12-b3e2-ee570cb3cbd4 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:55:26.845560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690428a-a175-42d3-b9a9-42b30ecae1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:55:26.846143 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690428a-a175-42d3-b9a9-42b30ecae1ec None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:55:26.846414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690428a-a175-42d3-b9a9-42b30ecae1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:55:26.846678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690428a-a175-42d3-b9a9-42b30ecae1ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:55:26.847417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690428a-a175-42d3-b9a9-42b30ecae1ec None admin] RBAC: Adding query filter params (role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.project.id=370228fbd935417b92a050cb09cc48de, user.id=33e65685d2e940a19d61603008366726) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:55:26.847717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690428a-a175-42d3-b9a9-42b30ecae1ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:55:26.849122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690428a-a175-42d3-b9a9-42b30ecae1ec None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:55:26.854619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 449/905] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:55:26 2022] GET /identity/v3/role_assignments?user.id=33e65685d2e940a19d61603008366726&scope.project.id=370228fbd935417b92a050cb09cc48de&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:55:36.702827 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=71 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:55:36.852701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=77 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:22.005069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa4521-b4c2-4a52-8afd-1f056e2aa8f8 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:22.005069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa4521-b4c2-4a52-8afd-1f056e2aa8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:22.005069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa4521-b4c2-4a52-8afd-1f056e2aa8f8 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:22.006156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 457/906] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:56:22.015680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20da8a8-50c1-4203-bd45-9e8aec8cff2b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:22.015680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20da8a8-50c1-4203-bd45-9e8aec8cff2b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:22.015680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20da8a8-50c1-4203-bd45-9e8aec8cff2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:22.034527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a20da8a8-50c1-4203-bd45-9e8aec8cff2b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:56:22.036040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a20da8a8-50c1-4203-bd45-9e8aec8cff2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:56:22.129706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 450/907] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:56:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:56:22.423821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa4521-b4c2-4a52-8afd-1f056e2aa8f8 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:22.569986 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa4521-b4c2-4a52-8afd-1f056e2aa8f8 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:22.610954 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102859a-9a6f-47f9-a955-14511f3415c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:22.611614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102859a-9a6f-47f9-a955-14511f3415c4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:22.611873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102859a-9a6f-47f9-a955-14511f3415c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:22.612138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102859a-9a6f-47f9-a955-14511f3415c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:22.626361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102859a-9a6f-47f9-a955-14511f3415c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:22.627848 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102859a-9a6f-47f9-a955-14511f3415c4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-9102859a-9a6f-47f9-a955-14511f3415c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:56:22.646637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:22.647877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:56:22.648863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:56:22.648863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:22.648863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:22.648863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 458/908] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:22.654066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20da8a8-50c1-4203-bd45-9e8aec8cff2b None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:22.804592 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20da8a8-50c1-4203-bd45-9e8aec8cff2b None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:22.841868 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:22.852767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dec36b-63e1-46a7-92ad-8a58979f8fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:22.853641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dec36b-63e1-46a7-92ad-8a58979f8fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:22.853918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dec36b-63e1-46a7-92ad-8a58979f8fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:22.854211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dec36b-63e1-46a7-92ad-8a58979f8fd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:22.854856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dec36b-63e1-46a7-92ad-8a58979f8fd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:22.855186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dec36b-63e1-46a7-92ad-8a58979f8fd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:22.856553 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dec36b-63e1-46a7-92ad-8a58979f8fd8 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:22.871210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 451/909] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:22.876250 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102859a-9a6f-47f9-a955-14511f3415c4 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:22.954696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:23.022478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102859a-9a6f-47f9-a955-14511f3415c4 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:23.066790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e59fa-f220-426a-a7a9-e8217236f7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:23.067718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e59fa-f220-426a-a7a9-e8217236f7ac None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:23.068020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e59fa-f220-426a-a7a9-e8217236f7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:23.068265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e59fa-f220-426a-a7a9-e8217236f7ac None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:23.069027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e59fa-f220-426a-a7a9-e8217236f7ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:23.071128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e59fa-f220-426a-a7a9-e8217236f7ac None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:23.099688 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 459/910] 10.222.0.58 () {60 vars in 1132 bytes} [Fri Oct 28 18:56:22 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:56:25.656913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe832b-f7c6-46b2-b302-0fcbabd33b3b None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:25.657557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe832b-f7c6-46b2-b302-0fcbabd33b3b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:25.657557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe832b-f7c6-46b2-b302-0fcbabd33b3b None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:25.659147 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 452/911] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:56:25.665733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3ed0-6d31-4f9c-ad9f-1faa7c01c2cc None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:25.666287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3ed0-6d31-4f9c-ad9f-1faa7c01c2cc None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:25.666287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3ed0-6d31-4f9c-ad9f-1faa7c01c2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:25.687153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2e3e3ed0-6d31-4f9c-ad9f-1faa7c01c2cc None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:56:25.688441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2e3e3ed0-6d31-4f9c-ad9f-1faa7c01c2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:56:25.764636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 460/912] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:56:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:56:26.035110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe832b-f7c6-46b2-b302-0fcbabd33b3b None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:26.149390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe832b-f7c6-46b2-b302-0fcbabd33b3b None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:26.186306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3eead1-6b1b-4489-9fe1-074b63739cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:26.187333 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eead1-6b1b-4489-9fe1-074b63739cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:26.187333 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eead1-6b1b-4489-9fe1-074b63739cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:26.187520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eead1-6b1b-4489-9fe1-074b63739cd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:26.198969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3eead1-6b1b-4489-9fe1-074b63739cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-1e3eead1-6b1b-4489-9fe1-074b63739cd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:56:26.204771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:26.206191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:56:26.207096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:26.210860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 453/913] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:56:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:26.213344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3ed0-6d31-4f9c-ad9f-1faa7c01c2cc None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:26.257291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:26.322810 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3ed0-6d31-4f9c-ad9f-1faa7c01c2cc None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:26.358171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cde939-4c5a-4963-96d2-63bcb3f86e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:26.358811 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cde939-4c5a-4963-96d2-63bcb3f86e9d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:26.359081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cde939-4c5a-4963-96d2-63bcb3f86e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:26.359493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cde939-4c5a-4963-96d2-63bcb3f86e9d None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:26.359989 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cde939-4c5a-4963-96d2-63bcb3f86e9d None admin] RBAC: Adding query filter params (name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:26.360295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cde939-4c5a-4963-96d2-63bcb3f86e9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:26.361644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cde939-4c5a-4963-96d2-63bcb3f86e9d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:26.370556 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 461/914] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:26.376272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3eead1-6b1b-4489-9fe1-074b63739cd1 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:26.436647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:26.505139 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3eead1-6b1b-4489-9fe1-074b63739cd1 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:26.552330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8377a-0cf5-45fa-b653-644f0278dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:26.553009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8377a-0cf5-45fa-b653-644f0278dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:26.553799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8377a-0cf5-45fa-b653-644f0278dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:26.554068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8377a-0cf5-45fa-b653-644f0278dfc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:26.566736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8377a-0cf5-45fa-b653-644f0278dfc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:26.568261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8377a-0cf5-45fa-b653-644f0278dfc6 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-e0e8377a-0cf5-45fa-b653-644f0278dfc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:56:26.585158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:26.586159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:56:26.587742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:56:26.587742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:26.587742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:26.587742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 454/915] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:26.592780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cde939-4c5a-4963-96d2-63bcb3f86e9d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:26.706791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:26.711116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cde939-4c5a-4963-96d2-63bcb3f86e9d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:26.756084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84b936-3b20-40be-8a14-c7bb47a2c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:26.757020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b936-3b20-40be-8a14-c7bb47a2c2ff None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:26.757535 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b936-3b20-40be-8a14-c7bb47a2c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:26.758069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b936-3b20-40be-8a14-c7bb47a2c2ff None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:26.758947 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84b936-3b20-40be-8a14-c7bb47a2c2ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:26.759566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84b936-3b20-40be-8a14-c7bb47a2c2ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:26.761231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84b936-3b20-40be-8a14-c7bb47a2c2ff None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:26.775237 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 462/916] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:26.781374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8377a-0cf5-45fa-b653-644f0278dfc6 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:26.902275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8377a-0cf5-45fa-b653-644f0278dfc6 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:26.903901 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:26.943266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea20e7-1bc6-41d7-b670-43ba3374541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:26.946027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea20e7-1bc6-41d7-b670-43ba3374541d None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:26.946316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea20e7-1bc6-41d7-b670-43ba3374541d None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:26.946658 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea20e7-1bc6-41d7-b670-43ba3374541d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:26.954659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea20e7-1bc6-41d7-b670-43ba3374541d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:26.956197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea20e7-1bc6-41d7-b670-43ba3374541d None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-98ea20e7-1bc6-41d7-b670-43ba3374541d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:56:26.967889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:26.968837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:26.969795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 455/917] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:26.975549 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84b936-3b20-40be-8a14-c7bb47a2c2ff None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:27.097716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84b936-3b20-40be-8a14-c7bb47a2c2ff None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:27.136382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95168ae-4351-4c4d-9f35-9140465081d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:27.136992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95168ae-4351-4c4d-9f35-9140465081d0 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:27.137265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95168ae-4351-4c4d-9f35-9140465081d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:27.137533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95168ae-4351-4c4d-9f35-9140465081d0 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:27.138261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95168ae-4351-4c4d-9f35-9140465081d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:27.138568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95168ae-4351-4c4d-9f35-9140465081d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:27.140026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95168ae-4351-4c4d-9f35-9140465081d0 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:27.163273 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 463/918] 10.222.0.58 () {56 vars in 1143 bytes} [Fri Oct 28 18:56:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:27.167602 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea20e7-1bc6-41d7-b670-43ba3374541d None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:27.291815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea20e7-1bc6-41d7-b670-43ba3374541d None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:27.331211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efbac6-78a6-487e-96d3-75e88da103f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:27.331868 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efbac6-78a6-487e-96d3-75e88da103f5 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:27.332144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efbac6-78a6-487e-96d3-75e88da103f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:27.332424 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efbac6-78a6-487e-96d3-75e88da103f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:27.347419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efbac6-78a6-487e-96d3-75e88da103f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:27.349090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efbac6-78a6-487e-96d3-75e88da103f5 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-16efbac6-78a6-487e-96d3-75e88da103f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:56:27.367652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:27.368675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:56:27.370298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:56:27.370298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:27.370298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:27.370298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 456/919] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:27.374855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95168ae-4351-4c4d-9f35-9140465081d0 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:27.399335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:27.496383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95168ae-4351-4c4d-9f35-9140465081d0 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:27.538586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94c869-cf03-40c7-897d-2407ea411799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:27.539228 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94c869-cf03-40c7-897d-2407ea411799 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:27.539505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94c869-cf03-40c7-897d-2407ea411799 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:27.539815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94c869-cf03-40c7-897d-2407ea411799 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:27.540612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94c869-cf03-40c7-897d-2407ea411799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:27.540968 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94c869-cf03-40c7-897d-2407ea411799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:27.542505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94c869-cf03-40c7-897d-2407ea411799 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:27.555201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 464/920] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:27.561512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efbac6-78a6-487e-96d3-75e88da103f5 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:27.589560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:27.699762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efbac6-78a6-487e-96d3-75e88da103f5 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:27.735333 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439ef75-db42-4f37-96bb-f8b581840644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:27.736073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439ef75-db42-4f37-96bb-f8b581840644 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:27.736314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439ef75-db42-4f37-96bb-f8b581840644 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:27.736578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439ef75-db42-4f37-96bb-f8b581840644 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:27.744283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439ef75-db42-4f37-96bb-f8b581840644 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:27.745962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439ef75-db42-4f37-96bb-f8b581840644 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-5439ef75-db42-4f37-96bb-f8b581840644 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:56:27.758210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:27.759402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:27.760305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:56:27.760305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:56:27.760305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:56:27.760305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:56:27.760305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:56:27.760305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:56:27.760305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:56:27.760305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:27.760305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 457/921] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:56:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:56:27.766932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94c869-cf03-40c7-897d-2407ea411799 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:27.846749 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:27.870776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94c869-cf03-40c7-897d-2407ea411799 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:27.907357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe814a3c-67bf-4da2-ac49-4d27e58853b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:27.908348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe814a3c-67bf-4da2-ac49-4d27e58853b4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:27.908805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe814a3c-67bf-4da2-ac49-4d27e58853b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:27.909274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe814a3c-67bf-4da2-ac49-4d27e58853b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:27.910452 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe814a3c-67bf-4da2-ac49-4d27e58853b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:27.910969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe814a3c-67bf-4da2-ac49-4d27e58853b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:27.914126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe814a3c-67bf-4da2-ac49-4d27e58853b4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:27.934330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 465/922] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:56:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:27.940282 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439ef75-db42-4f37-96bb-f8b581840644 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:28.018382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:28.048693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439ef75-db42-4f37-96bb-f8b581840644 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:28.085157 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec3731-dcd8-431b-be0e-878bacf62a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:28.085815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec3731-dcd8-431b-be0e-878bacf62a17 None admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:28.086094 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec3731-dcd8-431b-be0e-878bacf62a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:28.086367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec3731-dcd8-431b-be0e-878bacf62a17 None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/da958657ac3041848d24a76b76705489/roles/01b71dd1f22e41269c3caa83dea57b2e` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:28.101971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec3731-dcd8-431b-be0e-878bacf62a17 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=da958657ac3041848d24a76b76705489, role_id=01b71dd1f22e41269c3caa83dea57b2e)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:28.103439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec3731-dcd8-431b-be0e-878bacf62a17 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:28.116027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 458/923] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:56:27 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/da958657ac3041848d24a76b76705489/roles/01b71dd1f22e41269c3caa83dea57b2e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:56:30.313940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62849dd8-5e74-4c64-8746-18be908b1fbc None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:30.314509 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62849dd8-5e74-4c64-8746-18be908b1fbc None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:30.314509 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62849dd8-5e74-4c64-8746-18be908b1fbc None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:30.315862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 466/924] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:56:30.321879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152377d-8e84-4b6d-b4d5-4b9e3811c8aa None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:30.322522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152377d-8e84-4b6d-b4d5-4b9e3811c8aa None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:30.322522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152377d-8e84-4b6d-b4d5-4b9e3811c8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:30.343690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-e152377d-8e84-4b6d-b4d5-4b9e3811c8aa None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:56:30.345578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e152377d-8e84-4b6d-b4d5-4b9e3811c8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:56:30.433067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 459/925] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:56:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:56:30.713055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62849dd8-5e74-4c64-8746-18be908b1fbc None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:30.823557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62849dd8-5e74-4c64-8746-18be908b1fbc None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:30.860496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9252e45-2a69-4fa6-9ecf-3d0cefe93714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:30.861132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9252e45-2a69-4fa6-9ecf-3d0cefe93714 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:30.861408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9252e45-2a69-4fa6-9ecf-3d0cefe93714 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:30.861676 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9252e45-2a69-4fa6-9ecf-3d0cefe93714 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:30.868902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:30.873888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9252e45-2a69-4fa6-9ecf-3d0cefe93714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-a9252e45-2a69-4fa6-9ecf-3d0cefe93714 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:56:30.879503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:30.880582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:30.881596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 467/926] 10.222.0.58 () {56 vars in 1102 bytes} [Fri Oct 28 18:56:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:30.887390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152377d-8e84-4b6d-b4d5-4b9e3811c8aa None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:30.946276 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:30.995155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152377d-8e84-4b6d-b4d5-4b9e3811c8aa None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:31.032325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e06f8-dd7f-4956-bae3-5812dc7c7da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:31.032931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e06f8-dd7f-4956-bae3-5812dc7c7da2 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:31.033236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e06f8-dd7f-4956-bae3-5812dc7c7da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:31.033471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e06f8-dd7f-4956-bae3-5812dc7c7da2 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:31.034123 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e06f8-dd7f-4956-bae3-5812dc7c7da2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:31.034421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e06f8-dd7f-4956-bae3-5812dc7c7da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:31.035808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e06f8-dd7f-4956-bae3-5812dc7c7da2 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:31.044425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 460/927] 10.222.0.58 () {56 vars in 1103 bytes} [Fri Oct 28 18:56:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:31.051031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9252e45-2a69-4fa6-9ecf-3d0cefe93714 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:31.168762 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9252e45-2a69-4fa6-9ecf-3d0cefe93714 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:31.206511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db5e5c-a63d-4dc2-a6e6-f55dbf5ecff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:31.207170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db5e5c-a63d-4dc2-a6e6-f55dbf5ecff7 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:31.207451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db5e5c-a63d-4dc2-a6e6-f55dbf5ecff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:31.207710 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db5e5c-a63d-4dc2-a6e6-f55dbf5ecff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:31.217916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db5e5c-a63d-4dc2-a6e6-f55dbf5ecff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:31.219410 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db5e5c-a63d-4dc2-a6e6-f55dbf5ecff7 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-c3db5e5c-a63d-4dc2-a6e6-f55dbf5ecff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:56:31.234791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:31.235981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:56:31.237889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:56:31.237889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:31.237889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:31.237889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 468/928] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:31.242452 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e06f8-dd7f-4956-bae3-5812dc7c7da2 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:31.348392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e06f8-dd7f-4956-bae3-5812dc7c7da2 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:31.364525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:31.381831 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5445d6-31dc-42cf-af8d-d07a38bdac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:31.382431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5445d6-31dc-42cf-af8d-d07a38bdac69 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:31.382704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5445d6-31dc-42cf-af8d-d07a38bdac69 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:31.382972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5445d6-31dc-42cf-af8d-d07a38bdac69 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:31.383640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5445d6-31dc-42cf-af8d-d07a38bdac69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:31.383926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5445d6-31dc-42cf-af8d-d07a38bdac69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:31.385306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5445d6-31dc-42cf-af8d-d07a38bdac69 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:31.398698 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 461/929] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:31.404870 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db5e5c-a63d-4dc2-a6e6-f55dbf5ecff7 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:31.470240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:31.522176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db5e5c-a63d-4dc2-a6e6-f55dbf5ecff7 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:31.557178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119700e4-4e26-4f79-af22-fda5c0299bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:31.557803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119700e4-4e26-4f79-af22-fda5c0299bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:31.558080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119700e4-4e26-4f79-af22-fda5c0299bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:31.558347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119700e4-4e26-4f79-af22-fda5c0299bd6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:31.568010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119700e4-4e26-4f79-af22-fda5c0299bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:31.570056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119700e4-4e26-4f79-af22-fda5c0299bd6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-119700e4-4e26-4f79-af22-fda5c0299bd6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:56:31.583931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:31.585583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:31.586873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 469/930] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:31.592494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5445d6-31dc-42cf-af8d-d07a38bdac69 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:31.703182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5445d6-31dc-42cf-af8d-d07a38bdac69 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:31.743420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57afbc-1e89-49ee-8bda-aefd27f08c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:31.744022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57afbc-1e89-49ee-8bda-aefd27f08c6f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:31.744300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57afbc-1e89-49ee-8bda-aefd27f08c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:31.745729 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57afbc-1e89-49ee-8bda-aefd27f08c6f None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:31.745729 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57afbc-1e89-49ee-8bda-aefd27f08c6f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:31.745729 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57afbc-1e89-49ee-8bda-aefd27f08c6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:31.747002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57afbc-1e89-49ee-8bda-aefd27f08c6f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:31.767072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 462/931] 10.222.0.58 () {56 vars in 1143 bytes} [Fri Oct 28 18:56:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:31.773777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119700e4-4e26-4f79-af22-fda5c0299bd6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:31.879702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119700e4-4e26-4f79-af22-fda5c0299bd6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:31.891007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:31.918734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9baca3-b180-4caa-91a3-04835cc288f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:31.919843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9baca3-b180-4caa-91a3-04835cc288f4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:31.920265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9baca3-b180-4caa-91a3-04835cc288f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:31.920539 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9baca3-b180-4caa-91a3-04835cc288f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:32.005519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9baca3-b180-4caa-91a3-04835cc288f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:32.007562 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9baca3-b180-4caa-91a3-04835cc288f4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:32.139028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 470/932] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:56:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:56:32.144843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57afbc-1e89-49ee-8bda-aefd27f08c6f None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:32.184774 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:32.262517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57afbc-1e89-49ee-8bda-aefd27f08c6f None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:32.301495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11d832-8314-4119-9b2a-3d9dbadc62d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:32.302122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11d832-8314-4119-9b2a-3d9dbadc62d3 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:32.302400 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11d832-8314-4119-9b2a-3d9dbadc62d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:32.302693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11d832-8314-4119-9b2a-3d9dbadc62d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:32.312554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11d832-8314-4119-9b2a-3d9dbadc62d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:32.313994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11d832-8314-4119-9b2a-3d9dbadc62d3 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-bb11d832-8314-4119-9b2a-3d9dbadc62d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:56:32.329811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:32.330835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:56:32.331752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:56:32.331752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:32.331752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:32.331752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 463/933] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:32.337455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9baca3-b180-4caa-91a3-04835cc288f4 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:32.350396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:32.447125 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9baca3-b180-4caa-91a3-04835cc288f4 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:32.483567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f19272-06a9-4133-a48f-e588411c39bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:32.484177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f19272-06a9-4133-a48f-e588411c39bd None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:32.484454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f19272-06a9-4133-a48f-e588411c39bd None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:32.484722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f19272-06a9-4133-a48f-e588411c39bd None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:32.485359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f19272-06a9-4133-a48f-e588411c39bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:32.485661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f19272-06a9-4133-a48f-e588411c39bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:32.487113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f19272-06a9-4133-a48f-e588411c39bd None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:32.507015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 471/934] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:56:32.512072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11d832-8314-4119-9b2a-3d9dbadc62d3 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:32.608183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:32.624581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11d832-8314-4119-9b2a-3d9dbadc62d3 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:32.663156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c71fc0-4692-4845-a30a-84c724134db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:32.663800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71fc0-4692-4845-a30a-84c724134db8 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:32.664161 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71fc0-4692-4845-a30a-84c724134db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:32.664403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71fc0-4692-4845-a30a-84c724134db8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:32.671584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c71fc0-4692-4845-a30a-84c724134db8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:32.673082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c71fc0-4692-4845-a30a-84c724134db8 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-b9c71fc0-4692-4845-a30a-84c724134db8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:56:32.684672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:32.686073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:32.687212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:56:32.687212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:56:32.687212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:56:32.687212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:56:32.687212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:56:32.687212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:56:32.687212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:56:32.687212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:32.687212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 464/935] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:56:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:32.694719 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f19272-06a9-4133-a48f-e588411c39bd None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:32.774775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:32.809082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f19272-06a9-4133-a48f-e588411c39bd None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:32.842999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6c2e0-95c3-40b6-bb2a-9d3330622bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:32.843598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6c2e0-95c3-40b6-bb2a-9d3330622bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:32.843871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6c2e0-95c3-40b6-bb2a-9d3330622bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:32.844134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6c2e0-95c3-40b6-bb2a-9d3330622bf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:32.844809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6c2e0-95c3-40b6-bb2a-9d3330622bf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:32.845105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6c2e0-95c3-40b6-bb2a-9d3330622bf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:32.846505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6c2e0-95c3-40b6-bb2a-9d3330622bf4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:32.858644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 472/936] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:56:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:32.864608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c71fc0-4692-4845-a30a-84c724134db8 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:32.970298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c71fc0-4692-4845-a30a-84c724134db8 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:33.007865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746a631-3c90-4e98-8e8c-bfd57b10eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:33.008439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746a631-3c90-4e98-8e8c-bfd57b10eb98 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:33.008712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746a631-3c90-4e98-8e8c-bfd57b10eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:33.008976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746a631-3c90-4e98-8e8c-bfd57b10eb98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:33.009683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746a631-3c90-4e98-8e8c-bfd57b10eb98 None admin] RBAC: Adding query filter params (role.id=01b71dd1f22e41269c3caa83dea57b2e, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=da958657ac3041848d24a76b76705489) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:33.009982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746a631-3c90-4e98-8e8c-bfd57b10eb98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:33.011487 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746a631-3c90-4e98-8e8c-bfd57b10eb98 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:33.017241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 465/937] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:56:32 2022] GET /identity/v3/role_assignments?user.id=da958657ac3041848d24a76b76705489&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=01b71dd1f22e41269c3caa83dea57b2e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:35.198561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b2367-f8e4-46a3-8f6b-3b35300df329 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:35.199348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b2367-f8e4-46a3-8f6b-3b35300df329 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:35.199348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b2367-f8e4-46a3-8f6b-3b35300df329 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:35.200891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 473/938] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:56:35.206149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7852d8-7f32-41d7-a153-5e5d959a71c2 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:35.206587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7852d8-7f32-41d7-a153-5e5d959a71c2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:35.206677 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7852d8-7f32-41d7-a153-5e5d959a71c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:35.226361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-aa7852d8-7f32-41d7-a153-5e5d959a71c2 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:56:35.227699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-aa7852d8-7f32-41d7-a153-5e5d959a71c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:56:35.231379 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:35.295555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 466/939] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:56:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:56:35.563044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b2367-f8e4-46a3-8f6b-3b35300df329 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:35.677104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b2367-f8e4-46a3-8f6b-3b35300df329 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:35.705384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:35.716665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061baae-8ec9-464d-a509-279346ffe53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:35.717375 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061baae-8ec9-464d-a509-279346ffe53d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:35.717653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061baae-8ec9-464d-a509-279346ffe53d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:35.717924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061baae-8ec9-464d-a509-279346ffe53d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:35.729915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061baae-8ec9-464d-a509-279346ffe53d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-3061baae-8ec9-464d-a509-279346ffe53d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:56:35.734775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:35.735972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:35.736889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 474/940] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:56:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:56:35.742614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7852d8-7f32-41d7-a153-5e5d959a71c2 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:35.846783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7852d8-7f32-41d7-a153-5e5d959a71c2 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:35.890916 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365adf62-5590-411d-99c0-b8b0e7b1ae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:35.891783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365adf62-5590-411d-99c0-b8b0e7b1ae80 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:35.892133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365adf62-5590-411d-99c0-b8b0e7b1ae80 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:35.892580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365adf62-5590-411d-99c0-b8b0e7b1ae80 None admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:35.893703 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365adf62-5590-411d-99c0-b8b0e7b1ae80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:35.894148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365adf62-5590-411d-99c0-b8b0e7b1ae80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:35.895658 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365adf62-5590-411d-99c0-b8b0e7b1ae80 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:35.905695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 467/941] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:56:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:35.913367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061baae-8ec9-464d-a509-279346ffe53d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:36.029753 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061baae-8ec9-464d-a509-279346ffe53d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:36.064854 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad00b91-5017-4b4f-bcd5-0f913af79b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:36.065624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad00b91-5017-4b4f-bcd5-0f913af79b0e None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:36.066007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad00b91-5017-4b4f-bcd5-0f913af79b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:36.066360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad00b91-5017-4b4f-bcd5-0f913af79b0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:36.077408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad00b91-5017-4b4f-bcd5-0f913af79b0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:36.079013 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad00b91-5017-4b4f-bcd5-0f913af79b0e None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-5ad00b91-5017-4b4f-bcd5-0f913af79b0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:56:36.093153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:36.094290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:56:36.095580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:56:36.095580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:36.095580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:36.095580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 475/942] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:36.100731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365adf62-5590-411d-99c0-b8b0e7b1ae80 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:36.103613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:36.204373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365adf62-5590-411d-99c0-b8b0e7b1ae80 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:36.239732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2e86-c278-4f44-bdee-a1bb9fec03ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:36.241003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2e86-c278-4f44-bdee-a1bb9fec03ff None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:36.241223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2e86-c278-4f44-bdee-a1bb9fec03ff None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:36.241490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2e86-c278-4f44-bdee-a1bb9fec03ff None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:36.242179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f2e86-c278-4f44-bdee-a1bb9fec03ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:36.242496 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f2e86-c278-4f44-bdee-a1bb9fec03ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:36.244012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f2e86-c278-4f44-bdee-a1bb9fec03ff None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:36.257580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 468/943] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 18:56:36.263723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad00b91-5017-4b4f-bcd5-0f913af79b0e None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:36.311074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:36.371925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad00b91-5017-4b4f-bcd5-0f913af79b0e None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:36.410213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253385d1-6279-4b6a-ba6b-c0fdf905834d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:36.412551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253385d1-6279-4b6a-ba6b-c0fdf905834d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:36.413313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253385d1-6279-4b6a-ba6b-c0fdf905834d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:36.413910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253385d1-6279-4b6a-ba6b-c0fdf905834d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:36.426087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253385d1-6279-4b6a-ba6b-c0fdf905834d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:36.428289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253385d1-6279-4b6a-ba6b-c0fdf905834d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-253385d1-6279-4b6a-ba6b-c0fdf905834d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:56:36.441243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:36.442686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:36.443894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 476/944] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:36.448498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2e86-c278-4f44-bdee-a1bb9fec03ff None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:36.524102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:36.569700 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2e86-c278-4f44-bdee-a1bb9fec03ff None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:36.605315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8ed0c-b15c-4d21-8360-49098aaf961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:36.605951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8ed0c-b15c-4d21-8360-49098aaf961e None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:36.606227 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8ed0c-b15c-4d21-8360-49098aaf961e None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:36.606511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8ed0c-b15c-4d21-8360-49098aaf961e None admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:36.607237 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8ed0c-b15c-4d21-8360-49098aaf961e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:36.607535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8ed0c-b15c-4d21-8360-49098aaf961e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:36.609047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8ed0c-b15c-4d21-8360-49098aaf961e None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:36.629635 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 469/945] 10.222.0.58 () {56 vars in 1143 bytes} [Fri Oct 28 18:56:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:36.633445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253385d1-6279-4b6a-ba6b-c0fdf905834d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:36.757421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253385d1-6279-4b6a-ba6b-c0fdf905834d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:36.759422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:36.797777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb842703-706b-43c7-a69c-be3e0b0fdd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:36.798505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb842703-706b-43c7-a69c-be3e0b0fdd76 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:36.798840 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb842703-706b-43c7-a69c-be3e0b0fdd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:36.799179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb842703-706b-43c7-a69c-be3e0b0fdd76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:36.811114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb842703-706b-43c7-a69c-be3e0b0fdd76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:36.812545 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb842703-706b-43c7-a69c-be3e0b0fdd76 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-eb842703-706b-43c7-a69c-be3e0b0fdd76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:56:36.828031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:36.829291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:56:36.830419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:56:36.830419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:36.830419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:36.830419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 477/946] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 18:56:36.836641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8ed0c-b15c-4d21-8360-49098aaf961e None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:36.949995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8ed0c-b15c-4d21-8360-49098aaf961e None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:36.970654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:36.986654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1752da-3aed-4248-9132-9e288ee8d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:36.987399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1752da-3aed-4248-9132-9e288ee8d825 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:36.987710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1752da-3aed-4248-9132-9e288ee8d825 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:36.988044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1752da-3aed-4248-9132-9e288ee8d825 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:36.988784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1752da-3aed-4248-9132-9e288ee8d825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:36.989147 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1752da-3aed-4248-9132-9e288ee8d825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:36.991382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1752da-3aed-4248-9132-9e288ee8d825 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:37.003983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 470/947] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:37.010129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb842703-706b-43c7-a69c-be3e0b0fdd76 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:37.123296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb842703-706b-43c7-a69c-be3e0b0fdd76 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:37.166293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e96c7-8d64-49d7-b911-b8fb793bfd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:37.166973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e96c7-8d64-49d7-b911-b8fb793bfd50 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:37.167270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e96c7-8d64-49d7-b911-b8fb793bfd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:37.167541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e96c7-8d64-49d7-b911-b8fb793bfd50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:37.174137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e96c7-8d64-49d7-b911-b8fb793bfd50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:37.175611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e96c7-8d64-49d7-b911-b8fb793bfd50 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-a41e96c7-8d64-49d7-b911-b8fb793bfd50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:56:37.187295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:37.193735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:37.195032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:56:37.195032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:56:37.195032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:56:37.195032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:56:37.195032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:56:37.195032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:56:37.195032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:56:37.195032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:37.195032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 478/948] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:56:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:37.195595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1752da-3aed-4248-9132-9e288ee8d825 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:37.300499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1752da-3aed-4248-9132-9e288ee8d825 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:37.337585 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25394fda-31ff-4947-8adb-0472bd1536a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:37.338256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25394fda-31ff-4947-8adb-0472bd1536a1 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:37.338531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25394fda-31ff-4947-8adb-0472bd1536a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:37.338797 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25394fda-31ff-4947-8adb-0472bd1536a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:37.339521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25394fda-31ff-4947-8adb-0472bd1536a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:37.339812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25394fda-31ff-4947-8adb-0472bd1536a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:37.341663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25394fda-31ff-4947-8adb-0472bd1536a1 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:37.356490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 471/949] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:56:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:37.363773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e96c7-8d64-49d7-b911-b8fb793bfd50 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:37.383177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:37.474100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e96c7-8d64-49d7-b911-b8fb793bfd50 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:37.510058 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e6492-6f55-4442-8aef-495b4e6774a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:37.510719 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e6492-6f55-4442-8aef-495b4e6774a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:37.511158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e6492-6f55-4442-8aef-495b4e6774a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:37.511438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e6492-6f55-4442-8aef-495b4e6774a4 None admin] PATH_INFO: `/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/da958657ac3041848d24a76b76705489/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:37.513374 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e6492-6f55-4442-8aef-495b4e6774a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a389b8e27a4c468491201f54ef9614, user_id=da958657ac3041848d24a76b76705489, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:37.514913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e6492-6f55-4442-8aef-495b4e6774a4 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:37.528406 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 479/950] 10.222.0.58 () {58 vars in 1439 bytes} [Fri Oct 28 18:56:37 2022] PUT /identity/v3/projects/f4a389b8e27a4c468491201f54ef9614/users/da958657ac3041848d24a76b76705489/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 18:56:39.722340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93149c-54db-4b77-9347-b7ab2fa966ab None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:39.722889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93149c-54db-4b77-9347-b7ab2fa966ab None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:39.722889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93149c-54db-4b77-9347-b7ab2fa966ab None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:39.724942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 472/951] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:56:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:56:39.730707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07129bc0-b981-44fa-abd8-72358334d464 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:39.731201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07129bc0-b981-44fa-abd8-72358334d464 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:39.731395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07129bc0-b981-44fa-abd8-72358334d464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:39.749554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-07129bc0-b981-44fa-abd8-72358334d464 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:56:39.751153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-07129bc0-b981-44fa-abd8-72358334d464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:56:39.821452 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 480/952] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:56:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 18:56:40.122065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93149c-54db-4b77-9347-b7ab2fa966ab None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:40.170904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:40.239385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93149c-54db-4b77-9347-b7ab2fa966ab None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:40.279161 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361072e-fcc3-4919-b21d-932161cac6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:40.280018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361072e-fcc3-4919-b21d-932161cac6da None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:40.280296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361072e-fcc3-4919-b21d-932161cac6da None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:40.280673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361072e-fcc3-4919-b21d-932161cac6da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:40.292624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361072e-fcc3-4919-b21d-932161cac6da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-1361072e-fcc3-4919-b21d-932161cac6da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 18:56:40.298950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:40.299919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:40.300972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 473/953] 10.222.0.58 () {56 vars in 1096 bytes} [Fri Oct 28 18:56:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:40.307473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07129bc0-b981-44fa-abd8-72358334d464 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:40.310513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:40.423594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07129bc0-b981-44fa-abd8-72358334d464 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:40.468915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071036-927e-41cc-9697-6c4513dcc20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:40.469753 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071036-927e-41cc-9697-6c4513dcc20d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:40.470155 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071036-927e-41cc-9697-6c4513dcc20d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:40.470542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071036-927e-41cc-9697-6c4513dcc20d None admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:40.471366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88071036-927e-41cc-9697-6c4513dcc20d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:40.471763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88071036-927e-41cc-9697-6c4513dcc20d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:40.473280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88071036-927e-41cc-9697-6c4513dcc20d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:40.482559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 481/954] 10.222.0.58 () {56 vars in 1099 bytes} [Fri Oct 28 18:56:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:40.490391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361072e-fcc3-4919-b21d-932161cac6da None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:40.605102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:40.608564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361072e-fcc3-4919-b21d-932161cac6da None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:40.644992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a366-8bcf-417f-841a-58b618c125a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:40.645640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a366-8bcf-417f-841a-58b618c125a6 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:40.645913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a366-8bcf-417f-841a-58b618c125a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:40.646214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a366-8bcf-417f-841a-58b618c125a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:40.657380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6a366-8bcf-417f-841a-58b618c125a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:40.658858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6a366-8bcf-417f-841a-58b618c125a6 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-3fc6a366-8bcf-417f-841a-58b618c125a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:56:40.675788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:40.677004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:56:40.678731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:56:40.678731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:40.678731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:40.680234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 474/955] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:40.686166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071036-927e-41cc-9697-6c4513dcc20d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:40.754447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:40.791888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071036-927e-41cc-9697-6c4513dcc20d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:40.826666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa2887-3d41-4746-8487-cee3e1b1d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:40.827303 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2887-3d41-4746-8487-cee3e1b1d990 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:40.827580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2887-3d41-4746-8487-cee3e1b1d990 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:40.827846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2887-3d41-4746-8487-cee3e1b1d990 None admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:40.828485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa2887-3d41-4746-8487-cee3e1b1d990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:40.828784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa2887-3d41-4746-8487-cee3e1b1d990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:40.830157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa2887-3d41-4746-8487-cee3e1b1d990 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:40.843501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 482/956] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:40.849365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a366-8bcf-417f-841a-58b618c125a6 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:40.953833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a366-8bcf-417f-841a-58b618c125a6 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:40.987043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aad326-012b-4961-b2f2-9ce5052f7068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:40.987667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aad326-012b-4961-b2f2-9ce5052f7068 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:40.987941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aad326-012b-4961-b2f2-9ce5052f7068 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:40.988207 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aad326-012b-4961-b2f2-9ce5052f7068 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:40.996146 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aad326-012b-4961-b2f2-9ce5052f7068 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:40.997573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aad326-012b-4961-b2f2-9ce5052f7068 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-33aad326-012b-4961-b2f2-9ce5052f7068 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 18:56:41.009144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:41.010091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:41.011083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 475/957] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:41.016203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa2887-3d41-4746-8487-cee3e1b1d990 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:41.117413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa2887-3d41-4746-8487-cee3e1b1d990 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:41.142697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:41.152826 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac9c8c-5b33-47e4-810b-3b31e32e2ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:41.153429 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac9c8c-5b33-47e4-810b-3b31e32e2ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:41.153701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac9c8c-5b33-47e4-810b-3b31e32e2ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:41.153966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac9c8c-5b33-47e4-810b-3b31e32e2ef6 None admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:41.154683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac9c8c-5b33-47e4-810b-3b31e32e2ef6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:41.154997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac9c8c-5b33-47e4-810b-3b31e32e2ef6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:41.156407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac9c8c-5b33-47e4-810b-3b31e32e2ef6 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:41.173779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 483/958] 10.222.0.58 () {56 vars in 1143 bytes} [Fri Oct 28 18:56:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:41.179752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aad326-012b-4961-b2f2-9ce5052f7068 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:41.194974 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:41.277576 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aad326-012b-4961-b2f2-9ce5052f7068 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:41.309614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64819acc-6847-4492-8a2f-de71b39ad671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:41.310198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64819acc-6847-4492-8a2f-de71b39ad671 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:41.310474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64819acc-6847-4492-8a2f-de71b39ad671 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:41.310743 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64819acc-6847-4492-8a2f-de71b39ad671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:41.374499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64819acc-6847-4492-8a2f-de71b39ad671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:41.375949 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64819acc-6847-4492-8a2f-de71b39ad671 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:41.389555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:41.479310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 476/959] 10.222.0.58 () {58 vars in 1282 bytes} [Fri Oct 28 18:56:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 18:56:41.485220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac9c8c-5b33-47e4-810b-3b31e32e2ef6 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:41.580827 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac9c8c-5b33-47e4-810b-3b31e32e2ef6 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:41.612933 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd0dc9-f9f7-41e8-be5a-62bd990fa04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:41.613540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd0dc9-f9f7-41e8-be5a-62bd990fa04d None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:41.613811 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd0dc9-f9f7-41e8-be5a-62bd990fa04d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:41.614073 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd0dc9-f9f7-41e8-be5a-62bd990fa04d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:41.623648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd0dc9-f9f7-41e8-be5a-62bd990fa04d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:41.625057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd0dc9-f9f7-41e8-be5a-62bd990fa04d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-9bfd0dc9-f9f7-41e8-be5a-62bd990fa04d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 18:56:41.639707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:41.640622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 18:56:41.641642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 18:56:41.641642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 18:56:41.641642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:41.641642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 484/960] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 18:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:41.647433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64819acc-6847-4492-8a2f-de71b39ad671 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:41.741434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64819acc-6847-4492-8a2f-de71b39ad671 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:41.754582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:41.776681 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa3dc5-3506-4dda-b279-2ef96a68f9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:41.777276 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa3dc5-3506-4dda-b279-2ef96a68f9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:41.777573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa3dc5-3506-4dda-b279-2ef96a68f9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:41.777839 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa3dc5-3506-4dda-b279-2ef96a68f9c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:41.778497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa3dc5-3506-4dda-b279-2ef96a68f9c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:41.778794 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa3dc5-3506-4dda-b279-2ef96a68f9c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:41.780195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa3dc5-3506-4dda-b279-2ef96a68f9c7 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:41.790950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 477/961] 10.222.0.58 () {56 vars in 1109 bytes} [Fri Oct 28 18:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:41.796618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd0dc9-f9f7-41e8-be5a-62bd990fa04d None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:41.830937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:41.893311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd0dc9-f9f7-41e8-be5a-62bd990fa04d None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:41.925708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39ae6f-23f1-4f8c-b394-ece4350e62e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:41.926318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39ae6f-23f1-4f8c-b394-ece4350e62e3 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:41.926590 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39ae6f-23f1-4f8c-b394-ece4350e62e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:41.926939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39ae6f-23f1-4f8c-b394-ece4350e62e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:41.933755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39ae6f-23f1-4f8c-b394-ece4350e62e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:41.935191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39ae6f-23f1-4f8c-b394-ece4350e62e3 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application [None req-5b39ae6f-23f1-4f8c-b394-ece4350e62e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 18:56:41.946526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:41.947584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:41.948502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 18:56:41.948502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 18:56:41.948502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 18:56:41.948502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 18:56:41.948502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 18:56:41.948502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 18:56:41.948502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 18:56:41.948502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: ERROR keystone.server.flask.application Oct 28 18:56:41.948502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 485/962] 10.222.0.58 () {56 vars in 1111 bytes} [Fri Oct 28 18:56:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:41.953503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa3dc5-3506-4dda-b279-2ef96a68f9c7 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:42.050070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa3dc5-3506-4dda-b279-2ef96a68f9c7 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:42.082114 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dabdc3-1e14-41fc-98d2-18620733933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:42.082640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dabdc3-1e14-41fc-98d2-18620733933f None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:42.082916 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dabdc3-1e14-41fc-98d2-18620733933f None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:42.083208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dabdc3-1e14-41fc-98d2-18620733933f None admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:42.084382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dabdc3-1e14-41fc-98d2-18620733933f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:42.084683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dabdc3-1e14-41fc-98d2-18620733933f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:42.086096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dabdc3-1e14-41fc-98d2-18620733933f None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:42.097480 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 478/963] 10.222.0.58 () {56 vars in 1148 bytes} [Fri Oct 28 18:56:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:42.104004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39ae6f-23f1-4f8c-b394-ece4350e62e3 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:42.193339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:42.199448 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39ae6f-23f1-4f8c-b394-ece4350e62e3 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:42.231360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd40cd3-656f-4a48-bab4-0b6657155c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:42.231947 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd40cd3-656f-4a48-bab4-0b6657155c25 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:42.232222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd40cd3-656f-4a48-bab4-0b6657155c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:42.232491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd40cd3-656f-4a48-bab4-0b6657155c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:42.233197 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd40cd3-656f-4a48-bab4-0b6657155c25 None admin] RBAC: Adding query filter params (role.id=5b9587b54a454ed5b8a7e02ed7071c8f, scope.project.id=f4a389b8e27a4c468491201f54ef9614, user.id=da958657ac3041848d24a76b76705489) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:42.233502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd40cd3-656f-4a48-bab4-0b6657155c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:42.234905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd40cd3-656f-4a48-bab4-0b6657155c25 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:42.240465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 486/964] 10.222.0.58 () {56 vars in 1374 bytes} [Fri Oct 28 18:56:42 2022] GET /identity/v3/role_assignments?user.id=da958657ac3041848d24a76b76705489&scope.project.id=f4a389b8e27a4c468491201f54ef9614&role.id=5b9587b54a454ed5b8a7e02ed7071c8f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:44.366526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b986cc7-58e5-43be-8173-68e87bbd9280 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:44.367102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b986cc7-58e5-43be-8173-68e87bbd9280 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:44.367102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b986cc7-58e5-43be-8173-68e87bbd9280 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:44.368422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 479/965] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:56:44.373673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337101-b661-4d47-ba70-5689105c2347 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:44.374045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337101-b661-4d47-ba70-5689105c2347 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:44.374220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337101-b661-4d47-ba70-5689105c2347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:44.394767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8b337101-b661-4d47-ba70-5689105c2347 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:56:44.396922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8b337101-b661-4d47-ba70-5689105c2347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:56:44.477709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 487/966] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:56:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:56:44.753255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b986cc7-58e5-43be-8173-68e87bbd9280 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:44.783889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:44.851153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b986cc7-58e5-43be-8173-68e87bbd9280 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:44.883852 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33091a-da61-4fc0-a1ed-aa7340327083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:44.884467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33091a-da61-4fc0-a1ed-aa7340327083 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:44.884743 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33091a-da61-4fc0-a1ed-aa7340327083 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:44.885010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33091a-da61-4fc0-a1ed-aa7340327083 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:44.885579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33091a-da61-4fc0-a1ed-aa7340327083 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:44.886959 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33091a-da61-4fc0-a1ed-aa7340327083 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-1a33091a-da61-4fc0-a1ed-aa7340327083 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:44.895726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:44.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 18:56:44.898244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:44.901857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 480/967] 10.222.0.58 () {56 vars in 1117 bytes} [Fri Oct 28 18:56:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:44.902838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b337101-b661-4d47-ba70-5689105c2347 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:44.987112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:44.996890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b337101-b661-4d47-ba70-5689105c2347 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:45.029314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f477bb-6369-43a0-93d6-b29f893ba472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:45.029900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f477bb-6369-43a0-93d6-b29f893ba472 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:45.030175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f477bb-6369-43a0-93d6-b29f893ba472 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:45.030438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f477bb-6369-43a0-93d6-b29f893ba472 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:45.031085 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f477bb-6369-43a0-93d6-b29f893ba472 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:45.031386 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f477bb-6369-43a0-93d6-b29f893ba472 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:45.032734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f477bb-6369-43a0-93d6-b29f893ba472 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:45.037295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 488/968] 10.222.0.58 () {56 vars in 1116 bytes} [Fri Oct 28 18:56:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:45.042854 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33091a-da61-4fc0-a1ed-aa7340327083 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:45.133682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:45.139514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33091a-da61-4fc0-a1ed-aa7340327083 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:45.171613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7295ab-beba-45ab-aee9-9968c0b00999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:45.172202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7295ab-beba-45ab-aee9-9968c0b00999 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:45.172480 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7295ab-beba-45ab-aee9-9968c0b00999 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:45.172753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7295ab-beba-45ab-aee9-9968c0b00999 None admin] PATH_INFO: `/v3/services` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:45.173368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7295ab-beba-45ab-aee9-9968c0b00999 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:45.173670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7295ab-beba-45ab-aee9-9968c0b00999 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:45.175055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7295ab-beba-45ab-aee9-9968c0b00999 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:45.179890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 481/969] 10.222.0.58 () {56 vars in 1116 bytes} [Fri Oct 28 18:56:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:47.294536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f4aff-57ae-46a2-b03f-e48680116ef5 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:47.295118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f4aff-57ae-46a2-b03f-e48680116ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:47.295118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f4aff-57ae-46a2-b03f-e48680116ef5 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:47.296440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 489/970] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:56:47.301504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99876af2-c840-4b78-b113-ebcdbbe70c79 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:47.301889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99876af2-c840-4b78-b113-ebcdbbe70c79 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:47.302047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99876af2-c840-4b78-b113-ebcdbbe70c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:47.319727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-99876af2-c840-4b78-b113-ebcdbbe70c79 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:56:47.320901 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-99876af2-c840-4b78-b113-ebcdbbe70c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:56:47.385104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 482/971] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:56:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:56:47.655199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f4aff-57ae-46a2-b03f-e48680116ef5 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:47.750491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f4aff-57ae-46a2-b03f-e48680116ef5 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:47.783384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01e6a1-f9c0-4f4f-ba16-aeb113d1af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:47.783982 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01e6a1-f9c0-4f4f-ba16-aeb113d1af6d None admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:47.784258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01e6a1-f9c0-4f4f-ba16-aeb113d1af6d None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:47.784527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01e6a1-f9c0-4f4f-ba16-aeb113d1af6d None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:47.785178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01e6a1-f9c0-4f4f-ba16-aeb113d1af6d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:47.786521 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01e6a1-f9c0-4f4f-ba16-aeb113d1af6d None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:47.799804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 490/972] 10.222.0.58 () {60 vars in 1141 bytes} [Fri Oct 28 18:56:47 2022] POST /identity/v3/services => generated 246 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 18:56:49.923305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966eaff-94b4-404c-bd89-37339ef52ce7 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:49.923856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966eaff-94b4-404c-bd89-37339ef52ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:49.923856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966eaff-94b4-404c-bd89-37339ef52ce7 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:49.925160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 483/973] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:56:49.930819 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd961-6ca8-47eb-ae4f-bbe247ae8328 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:49.931222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd961-6ca8-47eb-ae4f-bbe247ae8328 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:49.931421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd961-6ca8-47eb-ae4f-bbe247ae8328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:49.949066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ef1bd961-6ca8-47eb-ae4f-bbe247ae8328 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:56:49.950243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ef1bd961-6ca8-47eb-ae4f-bbe247ae8328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:56:49.959697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:50.022166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 491/974] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:56:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:56:50.290698 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966eaff-94b4-404c-bd89-37339ef52ce7 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:50.377951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:50.389879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966eaff-94b4-404c-bd89-37339ef52ce7 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:50.427321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb54bee-7d65-41d9-88bc-33c2494c6834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:50.427986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb54bee-7d65-41d9-88bc-33c2494c6834 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:50.428269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb54bee-7d65-41d9-88bc-33c2494c6834 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:50.428574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb54bee-7d65-41d9-88bc-33c2494c6834 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:50.429222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb54bee-7d65-41d9-88bc-33c2494c6834 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:50.430726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb54bee-7d65-41d9-88bc-33c2494c6834 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-7cb54bee-7d65-41d9-88bc-33c2494c6834 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:50.440645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:50.442043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:50.442878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 484/975] 10.222.0.58 () {56 vars in 1117 bytes} [Fri Oct 28 18:56:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:50.448406 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd961-6ca8-47eb-ae4f-bbe247ae8328 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:50.547544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd961-6ca8-47eb-ae4f-bbe247ae8328 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:50.579259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178691-4357-491d-a6da-191dae2aa622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:50.579841 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178691-4357-491d-a6da-191dae2aa622 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:50.580113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178691-4357-491d-a6da-191dae2aa622 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:50.580375 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178691-4357-491d-a6da-191dae2aa622 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:50.580983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9178691-4357-491d-a6da-191dae2aa622 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:50.581284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9178691-4357-491d-a6da-191dae2aa622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:50.582617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9178691-4357-491d-a6da-191dae2aa622 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:50.587484 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 492/976] 10.222.0.58 () {56 vars in 1116 bytes} [Fri Oct 28 18:56:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:50.593534 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb54bee-7d65-41d9-88bc-33c2494c6834 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:50.692107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb54bee-7d65-41d9-88bc-33c2494c6834 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:50.724124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac25f3-48ba-4e42-8034-0d9aa6deaeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:50.725260 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac25f3-48ba-4e42-8034-0d9aa6deaeaa None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:50.725536 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac25f3-48ba-4e42-8034-0d9aa6deaeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:50.725804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac25f3-48ba-4e42-8034-0d9aa6deaeaa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:50.726491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac25f3-48ba-4e42-8034-0d9aa6deaeaa None admin] RBAC: Adding query filter params (interface=public, service_id=14688c08ba3c4e0dbe7278fb029fd1a3, region_id=RegionOne) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:50.726800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac25f3-48ba-4e42-8034-0d9aa6deaeaa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:50.728205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac25f3-48ba-4e42-8034-0d9aa6deaeaa None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:50.733178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 485/977] 10.222.0.58 () {56 vars in 1251 bytes} [Fri Oct 28 18:56:50 2022] GET /identity/v3/endpoints?service_id=14688c08ba3c4e0dbe7278fb029fd1a3&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:50.738832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178691-4357-491d-a6da-191dae2aa622 None admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:50.742526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:50.834787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178691-4357-491d-a6da-191dae2aa622 None admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:50.866079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da732ae0-3390-435a-84ac-ea6cf23d0d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:50.866661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da732ae0-3390-435a-84ac-ea6cf23d0d62 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:50.866934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da732ae0-3390-435a-84ac-ea6cf23d0d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:50.867233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da732ae0-3390-435a-84ac-ea6cf23d0d62 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:50.867820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da732ae0-3390-435a-84ac-ea6cf23d0d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:50.869154 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da732ae0-3390-435a-84ac-ea6cf23d0d62 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:50.875667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 493/978] 10.222.0.58 () {56 vars in 1087 bytes} [Fri Oct 28 18:56:50 2022] GET /identity/v3/services => generated 2191 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 18:56:52.995442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed5bce-a868-44af-8648-c05996b73531 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:52.996024 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed5bce-a868-44af-8648-c05996b73531 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:52.996024 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed5bce-a868-44af-8648-c05996b73531 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:52.997309 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 486/979] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:56:53.002595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752d0e-a777-4f06-a972-f54207bbf913 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:53.003012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752d0e-a777-4f06-a972-f54207bbf913 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:53.003112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752d0e-a777-4f06-a972-f54207bbf913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:53.021614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-eb752d0e-a777-4f06-a972-f54207bbf913 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:56:53.022798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-eb752d0e-a777-4f06-a972-f54207bbf913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:56:53.081206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:53.086903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 494/980] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:56:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 18:56:53.351652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed5bce-a868-44af-8648-c05996b73531 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:53.371177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:53.468163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed5bce-a868-44af-8648-c05996b73531 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:53.502201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507d2a6-ab5a-4d71-8f3f-ec6dbb1c3d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:53.502817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507d2a6-ab5a-4d71-8f3f-ec6dbb1c3d08 None admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:53.503118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507d2a6-ab5a-4d71-8f3f-ec6dbb1c3d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:53.503390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507d2a6-ab5a-4d71-8f3f-ec6dbb1c3d08 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:53.503966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507d2a6-ab5a-4d71-8f3f-ec6dbb1c3d08 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:53.505335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507d2a6-ab5a-4d71-8f3f-ec6dbb1c3d08 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application [None req-b507d2a6-ab5a-4d71-8f3f-ec6dbb1c3d08 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 18:56:53.513853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 18:56:53.515132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application with Lock( Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self._enter() Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application return self.creator() Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application created_value = creator( Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: ERROR keystone.server.flask.application Oct 28 18:56:53.516501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 487/981] 10.222.0.58 () {56 vars in 1117 bytes} [Fri Oct 28 18:56:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 18:56:53.521196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752d0e-a777-4f06-a972-f54207bbf913 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:53.614491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752d0e-a777-4f06-a972-f54207bbf913 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:53.646972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1110b-5a55-46db-bceb-a96136cdd479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:53.647581 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1110b-5a55-46db-bceb-a96136cdd479 None admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:53.647856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1110b-5a55-46db-bceb-a96136cdd479 None admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:53.648125 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1110b-5a55-46db-bceb-a96136cdd479 None admin] PATH_INFO: `/v3/services` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:53.648737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1110b-5a55-46db-bceb-a96136cdd479 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 18:56:53.649032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1110b-5a55-46db-bceb-a96136cdd479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:53.650371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1110b-5a55-46db-bceb-a96136cdd479 None admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:53.655243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 495/982] 10.222.0.58 () {56 vars in 1116 bytes} [Fri Oct 28 18:56:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 18:56:53.660967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507d2a6-ab5a-4d71-8f3f-ec6dbb1c3d08 None admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:56:53.739750 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:56:53.757514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507d2a6-ab5a-4d71-8f3f-ec6dbb1c3d08 None admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:56:53.791193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b5ef7-3e05-4e0e-a468-8dfca868e778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:56:53.791780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b5ef7-3e05-4e0e-a468-8dfca868e778 None admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:56:53.792052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b5ef7-3e05-4e0e-a468-8dfca868e778 None admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:56:53.792317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b5ef7-3e05-4e0e-a468-8dfca868e778 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:56:53.792965 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b5ef7-3e05-4e0e-a468-8dfca868e778 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:56:53.794317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b5ef7-3e05-4e0e-a468-8dfca868e778 None admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:56:53.812472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 488/983] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 18:56:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 18:57:03.655186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=69 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:57:03.808509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:57:57.562035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d8569-e1a1-45fe-ab9e-4abb9be96aaf None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:57:57.562946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d8569-e1a1-45fe-ab9e-4abb9be96aaf None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:57:57.563486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d8569-e1a1-45fe-ab9e-4abb9be96aaf None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:57:57.570849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31f5de-0f22-437e-a462-e0a2c27072b5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:57:57.571734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31f5de-0f22-437e-a462-e0a2c27072b5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:57:57.572211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31f5de-0f22-437e-a462-e0a2c27072b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:57:57.575279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 496/984] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:57:57 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:57:57.656341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2b31f5de-0f22-437e-a462-e0a2c27072b5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:57:57.658038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2b31f5de-0f22-437e-a462-e0a2c27072b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:57:57.795040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 489/985] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:57:58.093073 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bd2f-0e9c-4ea9-a265-e5e4dcfdfeb7 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:57:58.093364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bd2f-0e9c-4ea9-a265-e5e4dcfdfeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:57:58.093632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bd2f-0e9c-4ea9-a265-e5e4dcfdfeb7 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:57:58.095118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 497/986] 10.222.0.58 () {54 vars in 965 bytes} [Fri Oct 28 18:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:57:58.101466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660ac9f-e87b-481c-8ee0-4a88a926583b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:57:58.101743 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660ac9f-e87b-481c-8ee0-4a88a926583b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:57:58.102240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660ac9f-e87b-481c-8ee0-4a88a926583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:57:58.182561 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d660ac9f-e87b-481c-8ee0-4a88a926583b None None] MFA Rules not processed for user `9cffb662f4104e349c81c578bf66ab7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:57:58.183913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d660ac9f-e87b-481c-8ee0-4a88a926583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:57:58.204439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:57:58.262160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 490/987] 10.222.0.58 () {58 vars in 1062 bytes} [Fri Oct 28 18:57:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:57:58.272697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4bd2f-0e9c-4ea9-a265-e5e4dcfdfeb7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:57:58.274457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d5f4bd2f-0e9c-4ea9-a265-e5e4dcfdfeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:57:58.397363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4bd2f-0e9c-4ea9-a265-e5e4dcfdfeb7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:57:58.487939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52d922-1b87-4fcc-ae8a-e56ee1cb76dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:57:58.488894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52d922-1b87-4fcc-ae8a-e56ee1cb76dd service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:57:58.489335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52d922-1b87-4fcc-ae8a-e56ee1cb76dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:57:58.489757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52d922-1b87-4fcc-ae8a-e56ee1cb76dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:57:58.491888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ec52d922-1b87-4fcc-ae8a-e56ee1cb76dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:57:58.559792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:57:58.621887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52d922-1b87-4fcc-ae8a-e56ee1cb76dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:57:58.623532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52d922-1b87-4fcc-ae8a-e56ee1cb76dd service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:57:58.644133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 498/988] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 18:58:01.091060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b656c0b-5f8e-48ff-b403-0c48b4aff38e None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:01.091607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b656c0b-5f8e-48ff-b403-0c48b4aff38e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:01.091607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b656c0b-5f8e-48ff-b403-0c48b4aff38e None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:01.092957 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 491/989] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:01.099392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21617f-91a0-4578-b573-b144a5ce0bee None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:01.099897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21617f-91a0-4578-b573-b144a5ce0bee None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:01.100001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21617f-91a0-4578-b573-b144a5ce0bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:01.185553 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ea21617f-91a0-4578-b573-b144a5ce0bee None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:01.186752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ea21617f-91a0-4578-b573-b144a5ce0bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:01.295031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 499/990] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:58:01.577913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b656c0b-5f8e-48ff-b403-0c48b4aff38e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:01.579558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7b656c0b-5f8e-48ff-b403-0c48b4aff38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:01.686577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b656c0b-5f8e-48ff-b403-0c48b4aff38e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:01.757624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:01.760164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d524e4-4a77-4196-a236-49b64d47eed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:01.760947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d524e4-4a77-4196-a236-49b64d47eed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:01.761399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d524e4-4a77-4196-a236-49b64d47eed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:01.761714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d524e4-4a77-4196-a236-49b64d47eed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:01.763483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e5d524e4-4a77-4196-a236-49b64d47eed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:01.871660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d524e4-4a77-4196-a236-49b64d47eed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:01.873088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d524e4-4a77-4196-a236-49b64d47eed3 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:01.891119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 492/991] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:04.330845 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf2a14-9c84-40a3-aee2-61a353e9b7bc None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:04.331440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf2a14-9c84-40a3-aee2-61a353e9b7bc None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:04.331440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf2a14-9c84-40a3-aee2-61a353e9b7bc None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:04.332784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 500/992] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:04.338136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc95bf4-863c-4eee-86b6-32ac7622e13e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:04.338531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc95bf4-863c-4eee-86b6-32ac7622e13e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:04.338687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc95bf4-863c-4eee-86b6-32ac7622e13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:04.409677 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-dbc95bf4-863c-4eee-86b6-32ac7622e13e None None] MFA Rules not processed for user `27e08ff6bb314f25a262830c9d8000e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:04.410895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-dbc95bf4-863c-4eee-86b6-32ac7622e13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:04.461197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:04.505374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 493/993] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:04.787976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf2a14-9c84-40a3-aee2-61a353e9b7bc None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:04.789498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9dbf2a14-9c84-40a3-aee2-61a353e9b7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:04.799908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:04.890902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf2a14-9c84-40a3-aee2-61a353e9b7bc None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:04.964207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522177be-56c8-4ed3-92b2-bade17e4ce62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:04.964806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522177be-56c8-4ed3-92b2-bade17e4ce62 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:04.965080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522177be-56c8-4ed3-92b2-bade17e4ce62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:04.965344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522177be-56c8-4ed3-92b2-bade17e4ce62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:04.966915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-522177be-56c8-4ed3-92b2-bade17e4ce62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:05.027392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:05.073683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522177be-56c8-4ed3-92b2-bade17e4ce62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:05.075149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522177be-56c8-4ed3-92b2-bade17e4ce62 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:05.093262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 501/994] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:07.773791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cd67b-4710-4c48-9b14-f71e3b66cb04 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:07.774230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cd67b-4710-4c48-9b14-f71e3b66cb04 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:07.774320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cd67b-4710-4c48-9b14-f71e3b66cb04 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:07.775715 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 494/995] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:07.780922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489de6-cb7f-4544-afad-207ab2818d66 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:07.781298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489de6-cb7f-4544-afad-207ab2818d66 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:07.781461 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489de6-cb7f-4544-afad-207ab2818d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:07.845655 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-83489de6-cb7f-4544-afad-207ab2818d66 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:07.846836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-83489de6-cb7f-4544-afad-207ab2818d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:07.939466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 502/996] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:08.222908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cd67b-4710-4c48-9b14-f71e3b66cb04 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:08.224478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-480cd67b-4710-4c48-9b14-f71e3b66cb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:08.324914 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cd67b-4710-4c48-9b14-f71e3b66cb04 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:08.397569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f428e9-070f-4ad3-884d-8ea0a2de8ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:08.398210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f428e9-070f-4ad3-884d-8ea0a2de8ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:08.398717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f428e9-070f-4ad3-884d-8ea0a2de8ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:08.399134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f428e9-070f-4ad3-884d-8ea0a2de8ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:08.401319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-85f428e9-070f-4ad3-884d-8ea0a2de8ec8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:08.408839 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:08.507428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f428e9-070f-4ad3-884d-8ea0a2de8ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:08.508764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f428e9-070f-4ad3-884d-8ea0a2de8ec8 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:08.526237 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 495/997] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:11.356482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6490cb-2fcf-449f-8421-b26240eac9a5 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:11.357052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6490cb-2fcf-449f-8421-b26240eac9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:11.357052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6490cb-2fcf-449f-8421-b26240eac9a5 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:11.358387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 503/998] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:11.364288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f6bde-7c2d-49b5-88da-e005bc82403f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:11.364877 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f6bde-7c2d-49b5-88da-e005bc82403f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:11.364877 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f6bde-7c2d-49b5-88da-e005bc82403f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:11.441686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-df1f6bde-7c2d-49b5-88da-e005bc82403f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:11.442927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-df1f6bde-7c2d-49b5-88da-e005bc82403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:11.504533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:11.545509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 496/999] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:11.827026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6490cb-2fcf-449f-8421-b26240eac9a5 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:11.828625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-fa6490cb-2fcf-449f-8421-b26240eac9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:11.848270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:11.932489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6490cb-2fcf-449f-8421-b26240eac9a5 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:12.007171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04856f-3bf3-404e-9124-bebd318ce84c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:12.007688 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04856f-3bf3-404e-9124-bebd318ce84c service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:12.007977 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04856f-3bf3-404e-9124-bebd318ce84c service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:12.008226 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04856f-3bf3-404e-9124-bebd318ce84c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:12.009790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cc04856f-3bf3-404e-9124-bebd318ce84c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:12.079698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:12.117541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04856f-3bf3-404e-9124-bebd318ce84c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:12.118964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04856f-3bf3-404e-9124-bebd318ce84c service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:12.136677 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 504/1000] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:14.341346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4a60d-7c22-4693-8ef8-f11508bd3c60 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:14.341908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4a60d-7c22-4693-8ef8-f11508bd3c60 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:14.341908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4a60d-7c22-4693-8ef8-f11508bd3c60 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:14.343255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 497/1001] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:14.348454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3b46-f786-4483-b501-8ec5256e4ec1 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:14.348865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3b46-f786-4483-b501-8ec5256e4ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:14.348967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3b46-f786-4483-b501-8ec5256e4ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:14.427513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-b65b3b46-f786-4483-b501-8ec5256e4ec1 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:14.428732 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b65b3b46-f786-4483-b501-8ec5256e4ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:14.526160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 505/1002] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:14.805816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4a60d-7c22-4693-8ef8-f11508bd3c60 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:14.807464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7fb4a60d-7c22-4693-8ef8-f11508bd3c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:14.910802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4a60d-7c22-4693-8ef8-f11508bd3c60 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:14.985150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a21ec-b2cb-410d-8598-f9cffc653f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:14.985742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a21ec-b2cb-410d-8598-f9cffc653f13 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:14.986029 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a21ec-b2cb-410d-8598-f9cffc653f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:14.986276 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a21ec-b2cb-410d-8598-f9cffc653f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:14.988512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6f6a21ec-b2cb-410d-8598-f9cffc653f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:15.005421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:15.097751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a21ec-b2cb-410d-8598-f9cffc653f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:15.099215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a21ec-b2cb-410d-8598-f9cffc653f13 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:15.117812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 498/1003] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 18:58:17.331393 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa2934-6100-4891-9b92-0ac6d1fe97f3 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:17.331964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa2934-6100-4891-9b92-0ac6d1fe97f3 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:17.331964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa2934-6100-4891-9b92-0ac6d1fe97f3 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:17.333275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 506/1004] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:17.338566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34c0c-911a-42ad-a9b1-f1a49a6bcd88 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:17.338968 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34c0c-911a-42ad-a9b1-f1a49a6bcd88 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:17.339157 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34c0c-911a-42ad-a9b1-f1a49a6bcd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:17.415513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-c6b34c0c-911a-42ad-a9b1-f1a49a6bcd88 None None] MFA Rules not processed for user `27e08ff6bb314f25a262830c9d8000e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:17.416738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c6b34c0c-911a-42ad-a9b1-f1a49a6bcd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:17.485172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:17.518553 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 499/1005] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:17.809012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa2934-6100-4891-9b92-0ac6d1fe97f3 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:17.810618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9caa2934-6100-4891-9b92-0ac6d1fe97f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:17.843132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:17.917914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa2934-6100-4891-9b92-0ac6d1fe97f3 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:17.991402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b30ff0-ebcd-40e6-853a-31eb2bbe87d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:17.992012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b30ff0-ebcd-40e6-853a-31eb2bbe87d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:17.992289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b30ff0-ebcd-40e6-853a-31eb2bbe87d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:17.992559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b30ff0-ebcd-40e6-853a-31eb2bbe87d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:17.994149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-28b30ff0-ebcd-40e6-853a-31eb2bbe87d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:18.073194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:18.106358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b30ff0-ebcd-40e6-853a-31eb2bbe87d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:18.107837 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b30ff0-ebcd-40e6-853a-31eb2bbe87d1 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:18.125856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 507/1006] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:17 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:20.483519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a7401-db76-4d8b-9243-0bc084606503 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:20.484117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a7401-db76-4d8b-9243-0bc084606503 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:20.484117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a7401-db76-4d8b-9243-0bc084606503 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:20.485389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 500/1007] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:20.491447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34eed16-d17c-468f-a994-68ba7cbd7273 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:20.491861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34eed16-d17c-468f-a994-68ba7cbd7273 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:20.491977 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34eed16-d17c-468f-a994-68ba7cbd7273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:20.556442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a34eed16-d17c-468f-a994-68ba7cbd7273 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:20.557630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a34eed16-d17c-468f-a994-68ba7cbd7273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:20.651213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 508/1008] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:20.929956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a7401-db76-4d8b-9243-0bc084606503 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:20.931529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-597a7401-db76-4d8b-9243-0bc084606503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:21.031936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a7401-db76-4d8b-9243-0bc084606503 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:21.108169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad5288-1f06-4e8e-9433-eda47e235ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:21.108768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad5288-1f06-4e8e-9433-eda47e235ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:21.109049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad5288-1f06-4e8e-9433-eda47e235ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:21.109316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad5288-1f06-4e8e-9433-eda47e235ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:21.110905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7bad5288-1f06-4e8e-9433-eda47e235ff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:21.137405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:21.217576 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad5288-1f06-4e8e-9433-eda47e235ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:21.219012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad5288-1f06-4e8e-9433-eda47e235ff1 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:21.236621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 501/1009] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:23.431991 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865e043-bad7-4087-9f23-c0ff59ec10b0 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:23.432532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865e043-bad7-4087-9f23-c0ff59ec10b0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:23.432532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865e043-bad7-4087-9f23-c0ff59ec10b0 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:23.433864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 509/1010] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:23.439210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd748e-43cd-4ab9-8ba9-17d14d9e3725 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:23.439612 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd748e-43cd-4ab9-8ba9-17d14d9e3725 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:23.439763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd748e-43cd-4ab9-8ba9-17d14d9e3725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:23.504527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-c7cd748e-43cd-4ab9-8ba9-17d14d9e3725 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:23.505728 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c7cd748e-43cd-4ab9-8ba9-17d14d9e3725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:23.578508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:23.601523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 502/1011] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:23.886937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865e043-bad7-4087-9f23-c0ff59ec10b0 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:23.888545 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e865e043-bad7-4087-9f23-c0ff59ec10b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:23.928656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:23.991225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865e043-bad7-4087-9f23-c0ff59ec10b0 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:24.062464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af4f54-d45a-4ae8-82ad-4e3851cce628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:24.063097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af4f54-d45a-4ae8-82ad-4e3851cce628 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:24.063377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af4f54-d45a-4ae8-82ad-4e3851cce628 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:24.063648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af4f54-d45a-4ae8-82ad-4e3851cce628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:24.065766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-29af4f54-d45a-4ae8-82ad-4e3851cce628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:24.148228 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:24.173015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af4f54-d45a-4ae8-82ad-4e3851cce628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:24.174503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af4f54-d45a-4ae8-82ad-4e3851cce628 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:24.192537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 510/1012] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:26.619607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341437a-219b-45f5-9076-0b4b02b7b6b7 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:26.620143 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341437a-219b-45f5-9076-0b4b02b7b6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:26.620143 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341437a-219b-45f5-9076-0b4b02b7b6b7 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:26.621505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 503/1013] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:26.626730 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41732789-b378-480f-bccf-607a8cb70b60 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:26.627823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41732789-b378-480f-bccf-607a8cb70b60 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:26.627823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41732789-b378-480f-bccf-607a8cb70b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:26.700570 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-41732789-b378-480f-bccf-607a8cb70b60 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:26.701765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-41732789-b378-480f-bccf-607a8cb70b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:26.795709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 511/1014] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:27.076495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341437a-219b-45f5-9076-0b4b02b7b6b7 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:27.078008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0341437a-219b-45f5-9076-0b4b02b7b6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:27.178398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341437a-219b-45f5-9076-0b4b02b7b6b7 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:27.248360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c93f5d-8460-40a7-a64f-23468ade3300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:27.248949 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c93f5d-8460-40a7-a64f-23468ade3300 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:27.249219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c93f5d-8460-40a7-a64f-23468ade3300 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:27.249482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c93f5d-8460-40a7-a64f-23468ade3300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:27.251071 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-58c93f5d-8460-40a7-a64f-23468ade3300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:27.285572 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:27.356213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c93f5d-8460-40a7-a64f-23468ade3300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:27.357628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c93f5d-8460-40a7-a64f-23468ade3300 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:27.375675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 504/1015] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:31.379092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd87ff-e89a-4fdc-b69e-dbf60adc6f16 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:31.379691 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd87ff-e89a-4fdc-b69e-dbf60adc6f16 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:31.379691 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd87ff-e89a-4fdc-b69e-dbf60adc6f16 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:31.381206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 512/1016] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:31.386857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9dfac-91d0-4882-8596-1faa146d8069 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:31.387322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9dfac-91d0-4882-8596-1faa146d8069 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:31.387435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9dfac-91d0-4882-8596-1faa146d8069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:31.455788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-20c9dfac-91d0-4882-8596-1faa146d8069 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:31.457235 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-20c9dfac-91d0-4882-8596-1faa146d8069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:31.543279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:31.555247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 505/1017] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:31.857367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b025-87fa-4587-9b6e-254ca8e1b460 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:31.857738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b025-87fa-4587-9b6e-254ca8e1b460 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:31.858110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b025-87fa-4587-9b6e-254ca8e1b460 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:31.859561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 513/1018] 10.222.0.58 () {54 vars in 965 bytes} [Fri Oct 28 18:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:31.865668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f9e05-3573-4fe6-897e-13673a4ad3c6 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:31.865950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f9e05-3573-4fe6-897e-13673a4ad3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:31.866218 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f9e05-3573-4fe6-897e-13673a4ad3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:31.943551 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-441f9e05-3573-4fe6-897e-13673a4ad3c6 None None] MFA Rules not processed for user `9cffb662f4104e349c81c578bf66ab7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:31.944742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-441f9e05-3573-4fe6-897e-13673a4ad3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:32.019794 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 506/1019] 10.222.0.58 () {58 vars in 1062 bytes} [Fri Oct 28 18:58:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:58:32.028895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b025-87fa-4587-9b6e-254ca8e1b460 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:32.030532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-27c9b025-87fa-4587-9b6e-254ca8e1b460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:32.079424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:32.134335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b025-87fa-4587-9b6e-254ca8e1b460 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:32.208140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09632ea2-1158-411c-aea8-4f35b9823e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:32.208748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09632ea2-1158-411c-aea8-4f35b9823e29 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:32.209024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09632ea2-1158-411c-aea8-4f35b9823e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:32.209292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09632ea2-1158-411c-aea8-4f35b9823e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:32.210893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-09632ea2-1158-411c-aea8-4f35b9823e29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:32.309171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:32.327679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09632ea2-1158-411c-aea8-4f35b9823e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:32.329123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09632ea2-1158-411c-aea8-4f35b9823e29 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:32.349572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 514/1020] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:35.689478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5009e-f4dd-4593-a3ce-15ccc966faaf None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:35.689997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5009e-f4dd-4593-a3ce-15ccc966faaf None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:35.689997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5009e-f4dd-4593-a3ce-15ccc966faaf None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:35.691312 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 507/1021] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:35.696451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088885e-cfef-406c-849f-1de391bdd6fc None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:35.696914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088885e-cfef-406c-849f-1de391bdd6fc None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:35.697017 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088885e-cfef-406c-849f-1de391bdd6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:35.773455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-c088885e-cfef-406c-849f-1de391bdd6fc None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:35.774673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c088885e-cfef-406c-849f-1de391bdd6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:35.889030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 515/1022] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:36.173498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5009e-f4dd-4593-a3ce-15ccc966faaf None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:36.175122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c6a5009e-f4dd-4593-a3ce-15ccc966faaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:36.264145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:36.282379 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5009e-f4dd-4593-a3ce-15ccc966faaf None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:36.354397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643d0ce-a4e4-4ca1-8cd9-15c5d550fb3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:36.355017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643d0ce-a4e4-4ca1-8cd9-15c5d550fb3f service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:36.355298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643d0ce-a4e4-4ca1-8cd9-15c5d550fb3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:36.355565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643d0ce-a4e4-4ca1-8cd9-15c5d550fb3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:36.357150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d643d0ce-a4e4-4ca1-8cd9-15c5d550fb3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:36.466489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643d0ce-a4e4-4ca1-8cd9-15c5d550fb3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:36.467904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643d0ce-a4e4-4ca1-8cd9-15c5d550fb3f service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:36.487093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 508/1023] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:40.797478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b18b1b-8670-411a-8633-5e61f8ae9887 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:40.798033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b18b1b-8670-411a-8633-5e61f8ae9887 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:40.798033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b18b1b-8670-411a-8633-5e61f8ae9887 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:40.799420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 516/1024] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:40.804979 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14be811-344b-44d2-bd6c-c2f602c61602 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:40.805786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14be811-344b-44d2-bd6c-c2f602c61602 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:40.806232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14be811-344b-44d2-bd6c-c2f602c61602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:40.821941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:40.876240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a14be811-344b-44d2-bd6c-c2f602c61602 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:40.877970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a14be811-344b-44d2-bd6c-c2f602c61602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:40.976712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 509/1025] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:41.257653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b18b1b-8670-411a-8633-5e61f8ae9887 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:41.259197 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-69b18b1b-8670-411a-8633-5e61f8ae9887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:41.316603 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:41.360742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b18b1b-8670-411a-8633-5e61f8ae9887 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:41.436651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96b92e-57cb-4be2-a933-8f949f3663d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:41.437258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96b92e-57cb-4be2-a933-8f949f3663d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:41.437537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96b92e-57cb-4be2-a933-8f949f3663d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:41.437804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96b92e-57cb-4be2-a933-8f949f3663d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:41.439446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-fa96b92e-57cb-4be2-a933-8f949f3663d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:41.550414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:41.554290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96b92e-57cb-4be2-a933-8f949f3663d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:41.555919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96b92e-57cb-4be2-a933-8f949f3663d5 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:41.575075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 517/1026] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:46.479856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce4955-34ac-4203-b783-363e0e972b84 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:46.480302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce4955-34ac-4203-b783-363e0e972b84 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:46.480380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce4955-34ac-4203-b783-363e0e972b84 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:46.482473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 510/1027] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 18:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:46.488293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c9f1-f4ac-46d5-978b-ffea229a2911 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:46.488682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c9f1-f4ac-46d5-978b-ffea229a2911 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:46.488682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c9f1-f4ac-46d5-978b-ffea229a2911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:46.581314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2262c9f1-f4ac-46d5-978b-ffea229a2911 None None] MFA Rules not processed for user `da958657ac3041848d24a76b76705489`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:46.582897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2262c9f1-f4ac-46d5-978b-ffea229a2911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:46.765050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 518/1028] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 18:58:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:50.975168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=59 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:50.979988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdb3c0-69ef-4648-bba9-0ef4fdc8e42f None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:50.980477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdb3c0-69ef-4648-bba9-0ef4fdc8e42f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:50.981131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdb3c0-69ef-4648-bba9-0ef4fdc8e42f None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:50.983754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 511/1029] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 18:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:50.988237 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95172d4e-3a5f-45c1-a9aa-d79bf8ed011c None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:50.988706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95172d4e-3a5f-45c1-a9aa-d79bf8ed011c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:50.989046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95172d4e-3a5f-45c1-a9aa-d79bf8ed011c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:51.046219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:51.070907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-95172d4e-3a5f-45c1-a9aa-d79bf8ed011c None None] MFA Rules not processed for user `da958657ac3041848d24a76b76705489`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:51.072153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-95172d4e-3a5f-45c1-a9aa-d79bf8ed011c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:51.193517 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 519/1030] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 18:58:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:52.407160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f0427-1398-40d1-a36e-0b938f19d08d None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:52.408320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f0427-1398-40d1-a36e-0b938f19d08d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:52.408320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f0427-1398-40d1-a36e-0b938f19d08d None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:52.409592 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 512/1031] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 18:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:52.414646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704ec3c-00ae-4f5d-8991-2279f4cbf5ba None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:52.415147 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704ec3c-00ae-4f5d-8991-2279f4cbf5ba None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:52.415247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704ec3c-00ae-4f5d-8991-2279f4cbf5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:52.490499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e704ec3c-00ae-4f5d-8991-2279f4cbf5ba None None] MFA Rules not processed for user `da958657ac3041848d24a76b76705489`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:52.491720 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e704ec3c-00ae-4f5d-8991-2279f4cbf5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:52.533218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:52.598861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 520/1032] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 18:58:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:58:56.156803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba3c4-698a-49cc-9e27-cdb3c12effb0 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:56.157810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba3c4-698a-49cc-9e27-cdb3c12effb0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:56.158229 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba3c4-698a-49cc-9e27-cdb3c12effb0 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:56.160009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 513/1033] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:56.165646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22837fd0-95e6-4f27-a225-f4b6290deb56 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:56.166439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22837fd0-95e6-4f27-a225-f4b6290deb56 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:56.166884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22837fd0-95e6-4f27-a225-f4b6290deb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:56.237384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-22837fd0-95e6-4f27-a225-f4b6290deb56 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:56.238577 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-22837fd0-95e6-4f27-a225-f4b6290deb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:56.338412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 521/1034] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:58:56.641270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cc019-11e4-41e8-90d2-0523d6ea36f2 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:56.642663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cc019-11e4-41e8-90d2-0523d6ea36f2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:56.643192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cc019-11e4-41e8-90d2-0523d6ea36f2 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:56.645244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 514/1035] 10.222.0.58 () {54 vars in 961 bytes} [Fri Oct 28 18:58:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:56.655355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97abca9-9c8b-4cea-89ce-805dd8896f51 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:56.656243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97abca9-9c8b-4cea-89ce-805dd8896f51 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:56.656819 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97abca9-9c8b-4cea-89ce-805dd8896f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:56.679399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:56.748842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a97abca9-9c8b-4cea-89ce-805dd8896f51 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:56.750376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a97abca9-9c8b-4cea-89ce-805dd8896f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:56.874937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 522/1036] 10.222.0.58 () {58 vars in 1058 bytes} [Fri Oct 28 18:58:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:58:56.883523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cc019-11e4-41e8-90d2-0523d6ea36f2 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:58:56.885269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-ae6cc019-11e4-41e8-90d2-0523d6ea36f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:57.041011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cc019-11e4-41e8-90d2-0523d6ea36f2 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:58:57.117189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:57.158581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6843e-3bbb-4068-aa74-1a87943affc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:58:57.159206 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6843e-3bbb-4068-aa74-1a87943affc0 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:57.159404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6843e-3bbb-4068-aa74-1a87943affc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:57.159656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6843e-3bbb-4068-aa74-1a87943affc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:57.161230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b1d6843e-3bbb-4068-aa74-1a87943affc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:57.255526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6843e-3bbb-4068-aa74-1a87943affc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:58:57.257119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6843e-3bbb-4068-aa74-1a87943affc0 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:58:57.278631 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 515/1037] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:58:59.652260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a35a1-8495-48ad-b3e3-170bc8d0af85 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:59.652839 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a35a1-8495-48ad-b3e3-170bc8d0af85 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:59.652839 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a35a1-8495-48ad-b3e3-170bc8d0af85 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:59.654116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 523/1038] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:58:59.659573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aaffc-be7b-4224-9aa2-0f9c5d1e9380 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:58:59.660047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aaffc-be7b-4224-9aa2-0f9c5d1e9380 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:58:59.660614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aaffc-be7b-4224-9aa2-0f9c5d1e9380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:58:59.774832 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-406aaffc-be7b-4224-9aa2-0f9c5d1e9380 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:58:59.777326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-406aaffc-be7b-4224-9aa2-0f9c5d1e9380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:58:59.823791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:58:59.893669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 516/1039] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:00.186621 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a35a1-8495-48ad-b3e3-170bc8d0af85 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:00.189015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-213a35a1-8495-48ad-b3e3-170bc8d0af85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:00.265082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:00.368005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a35a1-8495-48ad-b3e3-170bc8d0af85 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:00.517941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6742da6-a6f0-4170-8d63-d74c5e376cad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:00.518828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742da6-a6f0-4170-8d63-d74c5e376cad service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:00.519148 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742da6-a6f0-4170-8d63-d74c5e376cad service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:00.519423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742da6-a6f0-4170-8d63-d74c5e376cad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:00.521024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a6742da6-a6f0-4170-8d63-d74c5e376cad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:00.544454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:00.615297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6742da6-a6f0-4170-8d63-d74c5e376cad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:00.617056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6742da6-a6f0-4170-8d63-d74c5e376cad service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:00.635833 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 524/1040] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:03.127903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a89d9-48cf-412f-ac78-23b705e10121 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:03.128465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a89d9-48cf-412f-ac78-23b705e10121 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:03.128465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a89d9-48cf-412f-ac78-23b705e10121 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:03.129978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 517/1041] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:03.135126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b87ae0-2f57-419d-906a-26d89ea70b70 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:03.135526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b87ae0-2f57-419d-906a-26d89ea70b70 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:03.135726 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b87ae0-2f57-419d-906a-26d89ea70b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:03.205363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-b9b87ae0-2f57-419d-906a-26d89ea70b70 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:03.206569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b9b87ae0-2f57-419d-906a-26d89ea70b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:03.306165 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:03.319399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 525/1042] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:03.596376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18c41-7c42-465c-9224-faf6443f33ca None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:03.596830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18c41-7c42-465c-9224-faf6443f33ca None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:03.597215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18c41-7c42-465c-9224-faf6443f33ca None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:03.598915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 518/1043] 10.222.0.58 () {54 vars in 961 bytes} [Fri Oct 28 18:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:03.604340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e697ea7-0a0c-4ca7-b4da-2bf8848638c1 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:03.604815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e697ea7-0a0c-4ca7-b4da-2bf8848638c1 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:03.605191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e697ea7-0a0c-4ca7-b4da-2bf8848638c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:03.684617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-4e697ea7-0a0c-4ca7-b4da-2bf8848638c1 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:03.685897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4e697ea7-0a0c-4ca7-b4da-2bf8848638c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:03.816531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 526/1044] 10.222.0.58 () {58 vars in 1058 bytes} [Fri Oct 28 18:59:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:59:03.825039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18c41-7c42-465c-9224-faf6443f33ca None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:03.826865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-bbf18c41-7c42-465c-9224-faf6443f33ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:03.981952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18c41-7c42-465c-9224-faf6443f33ca None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:04.005473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:04.106889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2c738-697a-4f53-8da7-ffb878ec1616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:04.107694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2c738-697a-4f53-8da7-ffb878ec1616 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:04.108107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2c738-697a-4f53-8da7-ffb878ec1616 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:04.108492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2c738-697a-4f53-8da7-ffb878ec1616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:04.110761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e9f2c738-697a-4f53-8da7-ffb878ec1616 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:04.204983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2c738-697a-4f53-8da7-ffb878ec1616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:04.206582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2c738-697a-4f53-8da7-ffb878ec1616 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:04.226206 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 519/1045] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:04.381283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf8de0-2c2b-469b-b9e2-a1a1577dffb6 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:04.381853 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf8de0-2c2b-469b-b9e2-a1a1577dffb6 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:04.381939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf8de0-2c2b-469b-b9e2-a1a1577dffb6 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:04.383522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 527/1046] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:04.389487 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331b733-82a3-4abf-adf3-f86567b9166b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:04.389778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331b733-82a3-4abf-adf3-f86567b9166b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:04.390213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331b733-82a3-4abf-adf3-f86567b9166b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:04.428118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:04.475330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3331b733-82a3-4abf-adf3-f86567b9166b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:04.476603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3331b733-82a3-4abf-adf3-f86567b9166b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:04.590690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 520/1047] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:04.928437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435d661-1b6d-4817-b272-9d72d8905c61 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:04.928803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435d661-1b6d-4817-b272-9d72d8905c61 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:04.929145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435d661-1b6d-4817-b272-9d72d8905c61 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:04.930604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 528/1048] 10.222.0.58 () {54 vars in 963 bytes} [Fri Oct 28 18:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:04.936548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48c99a-962b-4929-8ed8-439f065b752b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:04.936902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48c99a-962b-4929-8ed8-439f065b752b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:04.937238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48c99a-962b-4929-8ed8-439f065b752b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:05.020603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4b48c99a-962b-4929-8ed8-439f065b752b None None] MFA Rules not processed for user `302ded4fdd72405c8ce8a364f7d2400c`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:05.021987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4b48c99a-962b-4929-8ed8-439f065b752b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:05.070793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:05.102312 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 521/1049] 10.222.0.58 () {58 vars in 1060 bytes} [Fri Oct 28 18:59:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:59:05.110623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435d661-1b6d-4817-b272-9d72d8905c61 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:05.112163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d435d661-1b6d-4817-b272-9d72d8905c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:05.165633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:05.229027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435d661-1b6d-4817-b272-9d72d8905c61 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:05.313777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe47e-abf7-4c41-af1f-3138266f71a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:05.314415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe47e-abf7-4c41-af1f-3138266f71a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:05.314772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe47e-abf7-4c41-af1f-3138266f71a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:05.314924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe47e-abf7-4c41-af1f-3138266f71a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:05.316592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-663fe47e-abf7-4c41-af1f-3138266f71a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:05.428645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:05.443473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fe47e-abf7-4c41-af1f-3138266f71a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:05.445077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fe47e-abf7-4c41-af1f-3138266f71a9 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:05.466407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 529/1050] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:06.480365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ada4-f316-4fc2-b1ff-24f6434c4330 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:06.480896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ada4-f316-4fc2-b1ff-24f6434c4330 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:06.480896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ada4-f316-4fc2-b1ff-24f6434c4330 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:06.482155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 522/1051] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:59:06.488260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4ffb5-fda7-4db8-9d14-931eccc88835 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:06.488691 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4ffb5-fda7-4db8-9d14-931eccc88835 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:06.488913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4ffb5-fda7-4db8-9d14-931eccc88835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:06.554119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-cef4ffb5-fda7-4db8-9d14-931eccc88835 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:06.555336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cef4ffb5-fda7-4db8-9d14-931eccc88835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:06.651129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 530/1052] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:59:06.941874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99ada4-f316-4fc2-b1ff-24f6434c4330 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:06.943987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6b99ada4-f316-4fc2-b1ff-24f6434c4330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:07.101473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99ada4-f316-4fc2-b1ff-24f6434c4330 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:07.153961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:07.218195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06125b5e-4034-4527-b7e2-9f5f6331de3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:07.218797 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06125b5e-4034-4527-b7e2-9f5f6331de3a service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:07.219644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06125b5e-4034-4527-b7e2-9f5f6331de3a service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:07.219915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06125b5e-4034-4527-b7e2-9f5f6331de3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:07.221511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-06125b5e-4034-4527-b7e2-9f5f6331de3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:07.318387 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06125b5e-4034-4527-b7e2-9f5f6331de3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:07.320144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06125b5e-4034-4527-b7e2-9f5f6331de3a service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:07.338960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 523/1053] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:09.588312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102171e-aa60-45d1-8640-6457743c3924 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:09.588855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102171e-aa60-45d1-8640-6457743c3924 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:09.588855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102171e-aa60-45d1-8640-6457743c3924 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:09.590168 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 531/1054] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:09.595520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b20d9-1001-4ff5-9860-84f8d324ee09 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:09.595936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b20d9-1001-4ff5-9860-84f8d324ee09 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:09.596060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b20d9-1001-4ff5-9860-84f8d324ee09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:09.663541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-059b20d9-1001-4ff5-9860-84f8d324ee09 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:09.664771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-059b20d9-1001-4ff5-9860-84f8d324ee09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:09.672792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:09.788018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 524/1055] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:10.079783 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102171e-aa60-45d1-8640-6457743c3924 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:10.081538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8102171e-aa60-45d1-8640-6457743c3924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:10.145799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:10.235205 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102171e-aa60-45d1-8640-6457743c3924 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:10.354247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dacabe-5fe1-4356-a534-891bb7c9f8d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:10.354927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dacabe-5fe1-4356-a534-891bb7c9f8d5 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:10.355314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dacabe-5fe1-4356-a534-891bb7c9f8d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:10.355653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dacabe-5fe1-4356-a534-891bb7c9f8d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:10.357313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b6dacabe-5fe1-4356-a534-891bb7c9f8d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:10.377840 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:10.457042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dacabe-5fe1-4356-a534-891bb7c9f8d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:10.458781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dacabe-5fe1-4356-a534-891bb7c9f8d5 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:10.477835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 532/1056] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:12.700720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78d9e4-6e8f-4630-9381-b853b834aea2 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:12.701190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78d9e4-6e8f-4630-9381-b853b834aea2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:12.701271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78d9e4-6e8f-4630-9381-b853b834aea2 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:12.703106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 525/1057] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:12.707898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a1099-0e00-4f7e-9eea-9dd33aa8ca7e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:12.708297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a1099-0e00-4f7e-9eea-9dd33aa8ca7e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:12.708548 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a1099-0e00-4f7e-9eea-9dd33aa8ca7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:12.793059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-7d8a1099-0e00-4f7e-9eea-9dd33aa8ca7e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:12.794430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7d8a1099-0e00-4f7e-9eea-9dd33aa8ca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:12.891150 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:12.897732 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 533/1058] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:13.253453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78d9e4-6e8f-4630-9381-b853b834aea2 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:13.256028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7d78d9e4-6e8f-4630-9381-b853b834aea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:13.411876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:13.427346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78d9e4-6e8f-4630-9381-b853b834aea2 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:13.561380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d2c15-05ba-4833-98d0-d985458eca76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:13.561989 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d2c15-05ba-4833-98d0-d985458eca76 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:13.562265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d2c15-05ba-4833-98d0-d985458eca76 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:13.563206 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d2c15-05ba-4833-98d0-d985458eca76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:13.564736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6c4d2c15-05ba-4833-98d0-d985458eca76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:13.658006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d2c15-05ba-4833-98d0-d985458eca76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:13.659646 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d2c15-05ba-4833-98d0-d985458eca76 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:13.669808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:13.678559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 526/1059] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:15.873578 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5cc99-456e-4aea-9367-5acffbc16f0c None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:15.874129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5cc99-456e-4aea-9367-5acffbc16f0c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:15.874129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5cc99-456e-4aea-9367-5acffbc16f0c None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:15.875553 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 534/1060] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:15.881660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964725d-9277-4498-abb3-fdbe067b4cf6 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:15.882114 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964725d-9277-4498-abb3-fdbe067b4cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:15.882380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964725d-9277-4498-abb3-fdbe067b4cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:15.960375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-c964725d-9277-4498-abb3-fdbe067b4cf6 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:15.961774 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c964725d-9277-4498-abb3-fdbe067b4cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:16.064522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 527/1061] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:16.398164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5cc99-456e-4aea-9367-5acffbc16f0c None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:16.399740 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-dae5cc99-456e-4aea-9367-5acffbc16f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:16.583193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5cc99-456e-4aea-9367-5acffbc16f0c None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:16.663047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:16.741375 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57521b6e-814b-4467-be39-cc589444ee4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:16.742360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57521b6e-814b-4467-be39-cc589444ee4e service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:16.742774 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57521b6e-814b-4467-be39-cc589444ee4e service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:16.743298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57521b6e-814b-4467-be39-cc589444ee4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:16.745891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-57521b6e-814b-4467-be39-cc589444ee4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:16.850692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57521b6e-814b-4467-be39-cc589444ee4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:16.852400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57521b6e-814b-4467-be39-cc589444ee4e service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:16.870558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 535/1062] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:19.038801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5da55-8523-4e0e-b5a5-6ad6c9cc0a64 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:19.039468 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5da55-8523-4e0e-b5a5-6ad6c9cc0a64 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:19.039468 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5da55-8523-4e0e-b5a5-6ad6c9cc0a64 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:19.040732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 528/1063] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:19.046029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2dc09-c707-4168-97d8-7b36862760ca None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:19.046455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2dc09-c707-4168-97d8-7b36862760ca None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:19.047262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2dc09-c707-4168-97d8-7b36862760ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:19.113104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-30c2dc09-c707-4168-97d8-7b36862760ca None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:19.114295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-30c2dc09-c707-4168-97d8-7b36862760ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:19.147296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:19.213282 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 536/1064] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:19.510812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5da55-8523-4e0e-b5a5-6ad6c9cc0a64 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:19.512370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-34a5da55-8523-4e0e-b5a5-6ad6c9cc0a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:19.588968 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:19.658991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5da55-8523-4e0e-b5a5-6ad6c9cc0a64 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:19.774253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec8880-f0e6-4c78-9e9d-14af34321755 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:19.774854 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec8880-f0e6-4c78-9e9d-14af34321755 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:19.775158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec8880-f0e6-4c78-9e9d-14af34321755 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:19.775427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec8880-f0e6-4c78-9e9d-14af34321755 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:19.777025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-93ec8880-f0e6-4c78-9e9d-14af34321755 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:19.839534 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:19.892215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec8880-f0e6-4c78-9e9d-14af34321755 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:19.893786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec8880-f0e6-4c78-9e9d-14af34321755 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:19.911261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 529/1065] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:20.653750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8412342-495b-4325-a455-d589462a9ae9 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:20.654338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8412342-495b-4325-a455-d589462a9ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:20.654338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8412342-495b-4325-a455-d589462a9ae9 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:20.655714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 537/1066] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:20.661052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a0f5-b76f-4640-a816-0b49f224620f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:20.661481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a0f5-b76f-4640-a816-0b49f224620f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:20.661636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a0f5-b76f-4640-a816-0b49f224620f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:20.728045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2171a0f5-b76f-4640-a816-0b49f224620f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:20.729318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2171a0f5-b76f-4640-a816-0b49f224620f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:20.832709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 530/1067] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:21.134456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6948ed-d4e0-42b7-92c0-8c6d4d6b2589 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:21.134740 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6948ed-d4e0-42b7-92c0-8c6d4d6b2589 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:21.135040 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6948ed-d4e0-42b7-92c0-8c6d4d6b2589 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:21.136422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 538/1068] 10.222.0.58 () {54 vars in 962 bytes} [Fri Oct 28 18:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:21.141867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc4cdc-a794-401e-b06d-f28d3eb7bbe7 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:21.142148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc4cdc-a794-401e-b06d-f28d3eb7bbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:21.142419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc4cdc-a794-401e-b06d-f28d3eb7bbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:21.158727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:21.220077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-5acc4cdc-a794-401e-b06d-f28d3eb7bbe7 None None] MFA Rules not processed for user `6e8ec16bd9254b23bfd2b60551af9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:21.221307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5acc4cdc-a794-401e-b06d-f28d3eb7bbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:21.297052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 531/1069] 10.222.0.58 () {58 vars in 1059 bytes} [Fri Oct 28 18:59:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:21.305144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6948ed-d4e0-42b7-92c0-8c6d4d6b2589 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:21.306631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-bf6948ed-d4e0-42b7-92c0-8c6d4d6b2589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:21.410494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6948ed-d4e0-42b7-92c0-8c6d4d6b2589 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:21.470569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:21.486290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826f276-0b9c-4689-99f8-f19020ed5108 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8ec16bd9254b23bfd2b60551af9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:21.486892 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826f276-0b9c-4689-99f8-f19020ed5108 service glance] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:21.487191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826f276-0b9c-4689-99f8-f19020ed5108 service glance] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:21.487463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826f276-0b9c-4689-99f8-f19020ed5108 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:21.489029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9826f276-0b9c-4689-99f8-f19020ed5108 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:21.598145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826f276-0b9c-4689-99f8-f19020ed5108 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:21.599620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826f276-0b9c-4689-99f8-f19020ed5108 service glance] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:21.617362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 539/1070] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:22.105820 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fa42c-b399-4aa9-a4f8-89593cf756f2 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:22.106408 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fa42c-b399-4aa9-a4f8-89593cf756f2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:22.106408 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fa42c-b399-4aa9-a4f8-89593cf756f2 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:22.107725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 532/1071] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:22.113380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d67a49-02b0-4222-a346-9d9f199196ce None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:22.113791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d67a49-02b0-4222-a346-9d9f199196ce None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:22.113933 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d67a49-02b0-4222-a346-9d9f199196ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:22.188348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a8d67a49-02b0-4222-a346-9d9f199196ce None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:22.189598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a8d67a49-02b0-4222-a346-9d9f199196ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:22.235309 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:22.292894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 540/1072] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:22.359976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a998be-3e46-498f-94ce-021d2c790b74 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:22.360260 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a998be-3e46-498f-94ce-021d2c790b74 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:22.360534 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a998be-3e46-498f-94ce-021d2c790b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:22.429558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-59a998be-3e46-498f-94ce-021d2c790b74 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:22.430766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-59a998be-3e46-498f-94ce-021d2c790b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:22.484456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:22.516140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 533/1073] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 18:59:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:22.562513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423744b0-665f-4cd0-9d1d-4f8c4848c14f None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:22.562809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423744b0-665f-4cd0-9d1d-4f8c4848c14f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:22.563120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423744b0-665f-4cd0-9d1d-4f8c4848c14f None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:22.564500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 541/1074] 10.222.0.58 () {54 vars in 943 bytes} [Fri Oct 28 18:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:22.570797 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624e337-d5fe-46d3-a76d-758ec58b3220 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:22.571119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624e337-d5fe-46d3-a76d-758ec58b3220 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:22.571395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624e337-d5fe-46d3-a76d-758ec58b3220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:22.588464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423744b0-665f-4cd0-9d1d-4f8c4848c14f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:22.589960 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-423744b0-665f-4cd0-9d1d-4f8c4848c14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:22.648814 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4624e337-d5fe-46d3-a76d-758ec58b3220 None None] MFA Rules not processed for user `a51690d0367b45db8a6552764cc1d11e`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:22.650022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4624e337-d5fe-46d3-a76d-758ec58b3220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:22.740917 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423744b0-665f-4cd0-9d1d-4f8c4848c14f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:22.763205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 534/1075] 10.222.0.58 () {58 vars in 1040 bytes} [Fri Oct 28 18:59:22 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:22.773296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624e337-d5fe-46d3-a76d-758ec58b3220 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:22.773634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:22.784148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624e337-d5fe-46d3-a76d-758ec58b3220 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:22.810432 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:22.860286 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226596ef-a1c3-452c-b200-25f0dd0cec0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:22.861010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226596ef-a1c3-452c-b200-25f0dd0cec0c service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:22.861287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226596ef-a1c3-452c-b200-25f0dd0cec0c service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:22.861559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226596ef-a1c3-452c-b200-25f0dd0cec0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:22.863156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-226596ef-a1c3-452c-b200-25f0dd0cec0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:22.891867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075afb1-4aba-4773-879c-76e94612edb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:22.892469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075afb1-4aba-4773-879c-76e94612edb6 service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:22.892747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075afb1-4aba-4773-879c-76e94612edb6 service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:22.893015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075afb1-4aba-4773-879c-76e94612edb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:22.894649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3075afb1-4aba-4773-879c-76e94612edb6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:22.954202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226596ef-a1c3-452c-b200-25f0dd0cec0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:22.955808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226596ef-a1c3-452c-b200-25f0dd0cec0c service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:22.973411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 542/1076] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 18:59:22.983661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075afb1-4aba-4773-879c-76e94612edb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:22.985849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075afb1-4aba-4773-879c-76e94612edb6 service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:22.994314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 535/1077] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 18:59:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 28 18:59:25.217249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a018a-291f-444a-b880-5841b1f3a772 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:25.217829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a018a-291f-444a-b880-5841b1f3a772 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:25.217829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a018a-291f-444a-b880-5841b1f3a772 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:25.219736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 543/1078] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:25.224939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489164f5-5313-4dee-ad71-962a187fc507 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:25.225347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489164f5-5313-4dee-ad71-962a187fc507 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:25.225539 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489164f5-5313-4dee-ad71-962a187fc507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:25.296693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-489164f5-5313-4dee-ad71-962a187fc507 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:25.297930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-489164f5-5313-4dee-ad71-962a187fc507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:25.305220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:25.411264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 536/1079] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:25.704940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a018a-291f-444a-b880-5841b1f3a772 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:25.705909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f70a018a-291f-444a-b880-5841b1f3a772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:25.743090 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:25.873969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a018a-291f-444a-b880-5841b1f3a772 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:25.988394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:25.999561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756a940-5d2e-4546-8247-847ff538f5d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:26.000264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756a940-5d2e-4546-8247-847ff538f5d2 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:26.000599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756a940-5d2e-4546-8247-847ff538f5d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:26.000873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756a940-5d2e-4546-8247-847ff538f5d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:26.002456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1756a940-5d2e-4546-8247-847ff538f5d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:26.104189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756a940-5d2e-4546-8247-847ff538f5d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:26.106218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756a940-5d2e-4546-8247-847ff538f5d2 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:26.128696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 544/1080] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:28.351482 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300a8b6-3387-4e90-a86c-9cd365deda37 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:28.352020 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300a8b6-3387-4e90-a86c-9cd365deda37 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:28.352020 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300a8b6-3387-4e90-a86c-9cd365deda37 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:28.357042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 537/1081] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:28.358787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91f301-443d-4dc7-9f52-3042f9257c58 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:28.359750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91f301-443d-4dc7-9f52-3042f9257c58 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:28.360220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91f301-443d-4dc7-9f52-3042f9257c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:28.435008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-fe91f301-443d-4dc7-9f52-3042f9257c58 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:28.436707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-fe91f301-443d-4dc7-9f52-3042f9257c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:28.504247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:28.546338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 545/1082] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:59:28.842078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300a8b6-3387-4e90-a86c-9cd365deda37 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:28.843673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5300a8b6-3387-4e90-a86c-9cd365deda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:28.999945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:29.011087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300a8b6-3387-4e90-a86c-9cd365deda37 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:29.134486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8240d7-d017-4287-9329-245aa3c240f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:29.135119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8240d7-d017-4287-9329-245aa3c240f0 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:29.135392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8240d7-d017-4287-9329-245aa3c240f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:29.135656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8240d7-d017-4287-9329-245aa3c240f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:29.137245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-cc8240d7-d017-4287-9329-245aa3c240f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:29.235001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8240d7-d017-4287-9329-245aa3c240f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:29.236649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8240d7-d017-4287-9329-245aa3c240f0 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:29.247293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:29.256791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 538/1083] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:31.474065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943db47-a87e-4ca5-b057-f50d49bb48c7 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:31.474621 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943db47-a87e-4ca5-b057-f50d49bb48c7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:31.474621 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943db47-a87e-4ca5-b057-f50d49bb48c7 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:31.476186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 546/1084] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 18:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:31.481507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be59e0f-f3f0-4eec-9593-77007d89ebda None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:31.481977 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be59e0f-f3f0-4eec-9593-77007d89ebda None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:31.482082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be59e0f-f3f0-4eec-9593-77007d89ebda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:31.552476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3be59e0f-f3f0-4eec-9593-77007d89ebda None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:31.555017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3be59e0f-f3f0-4eec-9593-77007d89ebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:31.662667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 539/1085] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 18:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:31.954963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943db47-a87e-4ca5-b057-f50d49bb48c7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:31.956598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b943db47-a87e-4ca5-b057-f50d49bb48c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:32.126055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943db47-a87e-4ca5-b057-f50d49bb48c7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:32.181748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:32.253524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ecc75-c869-4700-ac45-a606a7064688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:32.254412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ecc75-c869-4700-ac45-a606a7064688 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:32.254690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ecc75-c869-4700-ac45-a606a7064688 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:32.254962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ecc75-c869-4700-ac45-a606a7064688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:32.258170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-443ecc75-c869-4700-ac45-a606a7064688 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:32.361016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ecc75-c869-4700-ac45-a606a7064688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:32.362611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ecc75-c869-4700-ac45-a606a7064688 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:32.381831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 547/1086] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:41.661230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=67 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:42.379438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=76 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:46.062576 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de43e1-4514-402f-a2c6-eda430ac6fc7 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:46.063170 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de43e1-4514-402f-a2c6-eda430ac6fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:46.063170 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de43e1-4514-402f-a2c6-eda430ac6fc7 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:46.064543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 540/1087] 10.222.0.58 () {54 vars in 918 bytes} [Fri Oct 28 18:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:46.069679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987ff8-4dfc-4f4b-a814-db9c312f8a0b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:46.070422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987ff8-4dfc-4f4b-a814-db9c312f8a0b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:46.070696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987ff8-4dfc-4f4b-a814-db9c312f8a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:46.146290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-1e987ff8-4dfc-4f4b-a814-db9c312f8a0b None None] MFA Rules not processed for user `da958657ac3041848d24a76b76705489`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:46.147513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1e987ff8-4dfc-4f4b-a814-db9c312f8a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:46.249601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 548/1088] 10.222.0.58 () {58 vars in 1015 bytes} [Fri Oct 28 18:59:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:46.263535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f8a39-f8d8-41ad-9bdb-3fc1fca2a1be None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:46.263815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f8a39-f8d8-41ad-9bdb-3fc1fca2a1be None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:46.264081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f8a39-f8d8-41ad-9bdb-3fc1fca2a1be None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:46.265386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 541/1089] 10.222.0.58 () {54 vars in 918 bytes} [Fri Oct 28 18:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:46.270052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b22c9c-c327-4826-9b8f-4f27627118b3 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:46.270329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b22c9c-c327-4826-9b8f-4f27627118b3 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:46.270595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b22c9c-c327-4826-9b8f-4f27627118b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:46.341719 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:46.344734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-94b22c9c-c327-4826-9b8f-4f27627118b3 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:46.346410 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-94b22c9c-c327-4826-9b8f-4f27627118b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:46.459949 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 549/1090] 10.222.0.58 () {58 vars in 1015 bytes} [Fri Oct 28 18:59:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:49.017487 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3c785-e784-424e-92b9-dd51f3f21c3e None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:49.018052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3c785-e784-424e-92b9-dd51f3f21c3e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:49.018052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3c785-e784-424e-92b9-dd51f3f21c3e None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:49.019431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 542/1091] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 18:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:49.024697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30b023-a9e7-4854-ab96-5d00610ee523 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:49.025098 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30b023-a9e7-4854-ab96-5d00610ee523 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:49.025236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30b023-a9e7-4854-ab96-5d00610ee523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:49.090858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ec30b023-a9e7-4854-ab96-5d00610ee523 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:49.092102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ec30b023-a9e7-4854-ab96-5d00610ee523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:49.145091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:49.185348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 550/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 18:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:49.470580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3c785-e784-424e-92b9-dd51f3f21c3e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:49.472256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3ba3c785-e784-424e-92b9-dd51f3f21c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:49.614547 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3c785-e784-424e-92b9-dd51f3f21c3e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:49.685501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:49.725174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a77db3-62ee-42e9-a1b3-32e7189adb2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:49.725701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a77db3-62ee-42e9-a1b3-32e7189adb2b service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:49.725974 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a77db3-62ee-42e9-a1b3-32e7189adb2b service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:49.726240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a77db3-62ee-42e9-a1b3-32e7189adb2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:49.727902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-32a77db3-62ee-42e9-a1b3-32e7189adb2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:49.818253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a77db3-62ee-42e9-a1b3-32e7189adb2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:49.819862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a77db3-62ee-42e9-a1b3-32e7189adb2b service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:49.837191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 543/1093] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:50.956885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4df86-f28f-42d7-9dce-4d13f865efcb None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:50.957443 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4df86-f28f-42d7-9dce-4d13f865efcb None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:50.957443 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4df86-f28f-42d7-9dce-4d13f865efcb None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:50.958744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 551/1094] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 18:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:50.963531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14156a-88d1-4f4c-b9ff-3eac0ac4ed31 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:50.963960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14156a-88d1-4f4c-b9ff-3eac0ac4ed31 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:50.964056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14156a-88d1-4f4c-b9ff-3eac0ac4ed31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:51.039360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-7c14156a-88d1-4f4c-b9ff-3eac0ac4ed31 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:51.040566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7c14156a-88d1-4f4c-b9ff-3eac0ac4ed31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:51.067560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:51.136185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 544/1095] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 18:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 18:59:51.290384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4df86-f28f-42d7-9dce-4d13f865efcb None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:51.291907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3ac4df86-f28f-42d7-9dce-4d13f865efcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:51.433749 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4df86-f28f-42d7-9dce-4d13f865efcb None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:51.473690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:51.545970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9b7f9-c8f3-4bb1-b123-8afae5f79504 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:51.546495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9b7f9-c8f3-4bb1-b123-8afae5f79504 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:51.546767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9b7f9-c8f3-4bb1-b123-8afae5f79504 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:51.547062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9b7f9-c8f3-4bb1-b123-8afae5f79504 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:51.548648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d4b9b7f9-c8f3-4bb1-b123-8afae5f79504 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:51.636945 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9b7f9-c8f3-4bb1-b123-8afae5f79504 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:51.638529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9b7f9-c8f3-4bb1-b123-8afae5f79504 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:51.656331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 552/1096] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:52.767377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd1da-8849-4b4a-b2bb-f458310ac532 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:52.767867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd1da-8849-4b4a-b2bb-f458310ac532 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:52.767972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd1da-8849-4b4a-b2bb-f458310ac532 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:52.769265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 545/1097] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 18:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:52.774285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762db2-9fb6-4849-9c9e-ee559df88a5d None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:52.774731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762db2-9fb6-4849-9c9e-ee559df88a5d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:52.774831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762db2-9fb6-4849-9c9e-ee559df88a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:52.828946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:52.851163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-48762db2-9fb6-4849-9c9e-ee559df88a5d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:52.852370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-48762db2-9fb6-4849-9c9e-ee559df88a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:52.946560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 553/1098] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 18:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:53.104304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0dd1da-8849-4b4a-b2bb-f458310ac532 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:53.105868 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6f0dd1da-8849-4b4a-b2bb-f458310ac532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:53.247055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0dd1da-8849-4b4a-b2bb-f458310ac532 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:53.258753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:53.357307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105b498-ae6d-45e2-91f4-c575653d9763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:53.357844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105b498-ae6d-45e2-91f4-c575653d9763 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:53.358120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105b498-ae6d-45e2-91f4-c575653d9763 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:53.358384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105b498-ae6d-45e2-91f4-c575653d9763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:53.359983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d105b498-ae6d-45e2-91f4-c575653d9763 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:53.449126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105b498-ae6d-45e2-91f4-c575653d9763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:53.450726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105b498-ae6d-45e2-91f4-c575653d9763 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:53.469443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 546/1099] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:54.561822 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014e39-0d96-4c88-8e22-228387a79c41 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:54.562466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014e39-0d96-4c88-8e22-228387a79c41 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:54.562466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014e39-0d96-4c88-8e22-228387a79c41 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:54.563720 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 554/1100] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 18:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 18:59:54.568734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806243f-6c54-4d44-94b7-2cc501f233f9 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:54.569177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806243f-6c54-4d44-94b7-2cc501f233f9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:54.569280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806243f-6c54-4d44-94b7-2cc501f233f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:54.588999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:54.644892 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4806243f-6c54-4d44-94b7-2cc501f233f9 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:54.646089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4806243f-6c54-4d44-94b7-2cc501f233f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:54.739928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 547/1101] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 18:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:54.889962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e014e39-0d96-4c88-8e22-228387a79c41 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:54.891753 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4e014e39-0d96-4c88-8e22-228387a79c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:54.999403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:55.034329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e014e39-0d96-4c88-8e22-228387a79c41 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:55.143179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08985b37-1377-4da1-80a3-9e9d9206df8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:55.143780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08985b37-1377-4da1-80a3-9e9d9206df8e service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:55.144050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08985b37-1377-4da1-80a3-9e9d9206df8e service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:55.144316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08985b37-1377-4da1-80a3-9e9d9206df8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:55.145890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-08985b37-1377-4da1-80a3-9e9d9206df8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:55.219378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:55.233189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08985b37-1377-4da1-80a3-9e9d9206df8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:55.234751 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08985b37-1377-4da1-80a3-9e9d9206df8e service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:55.252616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 555/1102] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:56.353149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81a974-92c1-42b5-b10f-eb80c1b963f9 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:56.353629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81a974-92c1-42b5-b10f-eb80c1b963f9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:56.353629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81a974-92c1-42b5-b10f-eb80c1b963f9 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:56.354993 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 548/1103] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 18:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:56.359953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87842f58-8794-4b7e-9655-1897fb022655 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:56.360381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87842f58-8794-4b7e-9655-1897fb022655 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:56.360502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87842f58-8794-4b7e-9655-1897fb022655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:56.436654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-87842f58-8794-4b7e-9655-1897fb022655 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:56.437853 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-87842f58-8794-4b7e-9655-1897fb022655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:56.532678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 556/1104] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 18:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:56.682609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81a974-92c1-42b5-b10f-eb80c1b963f9 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:56.684941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3c81a974-92c1-42b5-b10f-eb80c1b963f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:56.771178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:56.832073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81a974-92c1-42b5-b10f-eb80c1b963f9 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:56.942835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bab8bf-2590-4349-aad7-7a58fc19a5c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:56.943384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bab8bf-2590-4349-aad7-7a58fc19a5c4 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:56.943660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bab8bf-2590-4349-aad7-7a58fc19a5c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:56.943928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bab8bf-2590-4349-aad7-7a58fc19a5c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:56.945523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-23bab8bf-2590-4349-aad7-7a58fc19a5c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:56.994356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:57.035511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bab8bf-2590-4349-aad7-7a58fc19a5c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:57.037095 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bab8bf-2590-4349-aad7-7a58fc19a5c4 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:57.054478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 549/1105] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:58.083842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a02ecd-5372-4146-8647-cee55c281997 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:58.084360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a02ecd-5372-4146-8647-cee55c281997 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:58.084458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a02ecd-5372-4146-8647-cee55c281997 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:58.085739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 557/1106] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 18:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:58.090757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882db7e-0545-4651-8823-d545ff500773 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:58.091186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882db7e-0545-4651-8823-d545ff500773 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:58.091350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882db7e-0545-4651-8823-d545ff500773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:58.168102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-7882db7e-0545-4651-8823-d545ff500773 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:58.169313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7882db7e-0545-4651-8823-d545ff500773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:58.265053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 550/1107] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 18:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 18:59:58.415875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a02ecd-5372-4146-8647-cee55c281997 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 18:59:58.417272 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f8a02ecd-5372-4146-8647-cee55c281997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:58.467489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:58.561985 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a02ecd-5372-4146-8647-cee55c281997 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 18:59:58.671047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d06a21-4727-40ef-b648-a2d73e4bb75d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 18:59:58.671577 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06a21-4727-40ef-b648-a2d73e4bb75d service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:58.671857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06a21-4727-40ef-b648-a2d73e4bb75d service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:58.672123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06a21-4727-40ef-b648-a2d73e4bb75d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:58.673684 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-10d06a21-4727-40ef-b648-a2d73e4bb75d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:58.680350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 18:59:58.763870 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d06a21-4727-40ef-b648-a2d73e4bb75d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 18:59:58.765453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d06a21-4727-40ef-b648-a2d73e4bb75d service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 18:59:58.784016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 558/1108] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 18:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 18:59:59.832833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e66e4-4134-41ee-8905-0c8da15d7c87 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:59.833367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e66e4-4134-41ee-8905-0c8da15d7c87 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:59.833367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e66e4-4134-41ee-8905-0c8da15d7c87 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:59.834766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 551/1109] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 18:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 18:59:59.839707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3ec20-1377-45bf-9ac9-9031f061cb1e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 18:59:59.840142 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3ec20-1377-45bf-9ac9-9031f061cb1e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 18:59:59.840240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3ec20-1377-45bf-9ac9-9031f061cb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 18:59:59.914355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-36d3ec20-1377-45bf-9ac9-9031f061cb1e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 18:59:59.915567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-36d3ec20-1377-45bf-9ac9-9031f061cb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 18:59:59.981633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:00.012243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 559/1110] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 18:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:00.161063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e66e4-4134-41ee-8905-0c8da15d7c87 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:00.162634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e02e66e4-4134-41ee-8905-0c8da15d7c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:00.181397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:00.307655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e66e4-4134-41ee-8905-0c8da15d7c87 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:00.394122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:00.419117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848870ce-022d-4b40-9dfb-7c0fe269cd26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:00.420213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848870ce-022d-4b40-9dfb-7c0fe269cd26 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:00.420603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848870ce-022d-4b40-9dfb-7c0fe269cd26 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:00.420952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848870ce-022d-4b40-9dfb-7c0fe269cd26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:00.422870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-848870ce-022d-4b40-9dfb-7c0fe269cd26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:00.509144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848870ce-022d-4b40-9dfb-7c0fe269cd26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:00.510712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848870ce-022d-4b40-9dfb-7c0fe269cd26 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:00.528339 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 552/1111] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:01.568300 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df6745-5ef2-4406-9fdb-6754ed64bc23 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:01.568871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df6745-5ef2-4406-9fdb-6754ed64bc23 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:01.568871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df6745-5ef2-4406-9fdb-6754ed64bc23 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:01.570192 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 560/1112] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 19:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:01.575112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b9099-37d1-458f-a6f2-e640e5e4ee38 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:01.575485 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b9099-37d1-458f-a6f2-e640e5e4ee38 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:01.575654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b9099-37d1-458f-a6f2-e640e5e4ee38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:01.650291 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f21b9099-37d1-458f-a6f2-e640e5e4ee38 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:01.651512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f21b9099-37d1-458f-a6f2-e640e5e4ee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:01.697811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:01.752717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 553/1113] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:01.903362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df6745-5ef2-4406-9fdb-6754ed64bc23 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:01.905435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b4df6745-5ef2-4406-9fdb-6754ed64bc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:02.049823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df6745-5ef2-4406-9fdb-6754ed64bc23 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:02.102850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:02.160563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc482c6a-bd95-4a8f-8688-404f37987acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:02.161161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc482c6a-bd95-4a8f-8688-404f37987acd service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:02.161438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc482c6a-bd95-4a8f-8688-404f37987acd service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:02.161708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc482c6a-bd95-4a8f-8688-404f37987acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:02.163326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-fc482c6a-bd95-4a8f-8688-404f37987acd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:02.251188 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc482c6a-bd95-4a8f-8688-404f37987acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:02.252770 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc482c6a-bd95-4a8f-8688-404f37987acd service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:02.270793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 561/1114] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:03.310331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de127414-0e36-46be-a25b-cd249e368a91 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:03.310924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de127414-0e36-46be-a25b-cd249e368a91 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:03.310924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de127414-0e36-46be-a25b-cd249e368a91 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:03.312286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 554/1115] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 19:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:03.317195 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ddb04-c53d-447a-bd93-0e4ab92cc62a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:03.317646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ddb04-c53d-447a-bd93-0e4ab92cc62a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:03.317747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ddb04-c53d-447a-bd93-0e4ab92cc62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:03.391156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6b3ddb04-c53d-447a-bd93-0e4ab92cc62a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:03.392332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6b3ddb04-c53d-447a-bd93-0e4ab92cc62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:03.394291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:03.492253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 562/1116] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 19:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:03.642061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de127414-0e36-46be-a25b-cd249e368a91 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:03.643573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-de127414-0e36-46be-a25b-cd249e368a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:03.786846 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de127414-0e36-46be-a25b-cd249e368a91 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:03.814720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:03.900634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a33f7c-5cb9-49c0-a250-b8b823b55549 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:03.901178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a33f7c-5cb9-49c0-a250-b8b823b55549 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:03.901449 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a33f7c-5cb9-49c0-a250-b8b823b55549 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:03.901711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a33f7c-5cb9-49c0-a250-b8b823b55549 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:03.903313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b2a33f7c-5cb9-49c0-a250-b8b823b55549 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:03.995641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a33f7c-5cb9-49c0-a250-b8b823b55549 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:03.997205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a33f7c-5cb9-49c0-a250-b8b823b55549 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:04.016731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 555/1117] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:06.067647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51221d5c-69ef-4c14-9d3f-184cb6fbb195 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:06.068252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51221d5c-69ef-4c14-9d3f-184cb6fbb195 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:06.068252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51221d5c-69ef-4c14-9d3f-184cb6fbb195 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:06.069524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 563/1118] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:06.074718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91845026-1c36-4870-b9db-27a12c600de2 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:06.075168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91845026-1c36-4870-b9db-27a12c600de2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:06.075325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91845026-1c36-4870-b9db-27a12c600de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:06.115192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:06.141999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-91845026-1c36-4870-b9db-27a12c600de2 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:06.143252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-91845026-1c36-4870-b9db-27a12c600de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:06.237701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 556/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:06.522724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51221d5c-69ef-4c14-9d3f-184cb6fbb195 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:06.524278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-51221d5c-69ef-4c14-9d3f-184cb6fbb195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:06.624720 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51221d5c-69ef-4c14-9d3f-184cb6fbb195 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:06.657287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:06.698112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0964b-25a1-404d-9935-2ea0afd866e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:06.698711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0964b-25a1-404d-9935-2ea0afd866e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:06.699003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0964b-25a1-404d-9935-2ea0afd866e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:06.699667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0964b-25a1-404d-9935-2ea0afd866e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:06.701555 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-32c0964b-25a1-404d-9935-2ea0afd866e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:06.812893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0964b-25a1-404d-9935-2ea0afd866e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:06.814243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0964b-25a1-404d-9935-2ea0afd866e8 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:06.832383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 564/1120] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:09.182760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a032cc-13d8-4d7b-9f79-066171302da3 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:09.183371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a032cc-13d8-4d7b-9f79-066171302da3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:09.183371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a032cc-13d8-4d7b-9f79-066171302da3 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:09.185200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 557/1121] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:09.190326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41018e5e-2d10-491c-b057-c9d4fd7d7994 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:09.190733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41018e5e-2d10-491c-b057-c9d4fd7d7994 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:09.190873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41018e5e-2d10-491c-b057-c9d4fd7d7994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:09.255470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-41018e5e-2d10-491c-b057-c9d4fd7d7994 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:09.256669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-41018e5e-2d10-491c-b057-c9d4fd7d7994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:09.270102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:09.353819 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 565/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:09.642323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a032cc-13d8-4d7b-9f79-066171302da3 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:09.643899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b6a032cc-13d8-4d7b-9f79-066171302da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:09.746628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a032cc-13d8-4d7b-9f79-066171302da3 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:09.759874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:09.820232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e360c0-4f6b-41c8-8983-2574e188be42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:09.820823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e360c0-4f6b-41c8-8983-2574e188be42 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:09.821098 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e360c0-4f6b-41c8-8983-2574e188be42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:09.821361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e360c0-4f6b-41c8-8983-2574e188be42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:09.822935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f4e360c0-4f6b-41c8-8983-2574e188be42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:09.929639 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e360c0-4f6b-41c8-8983-2574e188be42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:09.931089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e360c0-4f6b-41c8-8983-2574e188be42 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:09.949744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 558/1123] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:12.399571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286921f1-4509-47d3-bde1-18ee7715bf94 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:12.400149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286921f1-4509-47d3-bde1-18ee7715bf94 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:12.400149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286921f1-4509-47d3-bde1-18ee7715bf94 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:12.401464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 566/1124] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:12.406941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8411aec-9994-4034-954d-c64d78f08109 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:12.407419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8411aec-9994-4034-954d-c64d78f08109 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:12.407515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8411aec-9994-4034-954d-c64d78f08109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:12.457951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:12.475967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-e8411aec-9994-4034-954d-c64d78f08109 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:12.477162 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e8411aec-9994-4034-954d-c64d78f08109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:12.571546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 559/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:12.858553 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286921f1-4509-47d3-bde1-18ee7715bf94 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:12.860158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-286921f1-4509-47d3-bde1-18ee7715bf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:12.962449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286921f1-4509-47d3-bde1-18ee7715bf94 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:13.002517 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:13.037285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb127892-2d05-4b2f-a65e-f639ab10c292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:13.037893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127892-2d05-4b2f-a65e-f639ab10c292 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:13.038174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127892-2d05-4b2f-a65e-f639ab10c292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:13.038454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127892-2d05-4b2f-a65e-f639ab10c292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:13.040107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cb127892-2d05-4b2f-a65e-f639ab10c292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:13.148701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb127892-2d05-4b2f-a65e-f639ab10c292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:13.150189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb127892-2d05-4b2f-a65e-f639ab10c292 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:13.167564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 567/1126] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:15.432290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb1b08-d60d-4990-9de2-20134a08a25a None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:15.432857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb1b08-d60d-4990-9de2-20134a08a25a None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:15.432857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb1b08-d60d-4990-9de2-20134a08a25a None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:15.434143 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 560/1127] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:15.439540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31d4d0-bb7e-4c4b-a046-52e276b98b1f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:15.439935 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31d4d0-bb7e-4c4b-a046-52e276b98b1f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:15.440105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31d4d0-bb7e-4c4b-a046-52e276b98b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:15.504956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ce31d4d0-bb7e-4c4b-a046-52e276b98b1f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:15.506163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ce31d4d0-bb7e-4c4b-a046-52e276b98b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:15.526969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:15.601919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 568/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:15.888028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb1b08-d60d-4990-9de2-20134a08a25a None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:15.890249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-16bb1b08-d60d-4990-9de2-20134a08a25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:15.992303 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb1b08-d60d-4990-9de2-20134a08a25a None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:16.013176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:16.066769 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379db081-ddff-4cc7-a02f-4f7b307e24c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:16.067394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379db081-ddff-4cc7-a02f-4f7b307e24c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:16.067672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379db081-ddff-4cc7-a02f-4f7b307e24c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:16.067940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379db081-ddff-4cc7-a02f-4f7b307e24c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:16.069527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-379db081-ddff-4cc7-a02f-4f7b307e24c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:16.177066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379db081-ddff-4cc7-a02f-4f7b307e24c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:16.178483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379db081-ddff-4cc7-a02f-4f7b307e24c6 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:16.196549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 561/1129] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:18.412386 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40a9c-531d-4a10-b73a-7c2048fe5c01 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:18.412978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40a9c-531d-4a10-b73a-7c2048fe5c01 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:18.412978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40a9c-531d-4a10-b73a-7c2048fe5c01 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:18.414282 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 569/1130] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:18.419644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f21d3-811e-4bd4-b214-41b14af9e95f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:18.420086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f21d3-811e-4bd4-b214-41b14af9e95f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:18.420196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f21d3-811e-4bd4-b214-41b14af9e95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:18.487867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:18.490858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a98f21d3-811e-4bd4-b214-41b14af9e95f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:18.492421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a98f21d3-811e-4bd4-b214-41b14af9e95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:18.587043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 562/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:18.872161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c40a9c-531d-4a10-b73a-7c2048fe5c01 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:18.873706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-85c40a9c-531d-4a10-b73a-7c2048fe5c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:18.975940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c40a9c-531d-4a10-b73a-7c2048fe5c01 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:19.024491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:19.049496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de60131-2671-4979-8f98-dc6f0bfb3c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:19.050098 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de60131-2671-4979-8f98-dc6f0bfb3c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:19.050375 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de60131-2671-4979-8f98-dc6f0bfb3c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:19.050650 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de60131-2671-4979-8f98-dc6f0bfb3c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:19.052268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3de60131-2671-4979-8f98-dc6f0bfb3c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:19.158630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de60131-2671-4979-8f98-dc6f0bfb3c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:19.160068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de60131-2671-4979-8f98-dc6f0bfb3c2a service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:19.178148 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 570/1132] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:21.366405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff8784-a668-4db8-b0de-c067f01e7be2 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:21.366970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff8784-a668-4db8-b0de-c067f01e7be2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:21.366970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff8784-a668-4db8-b0de-c067f01e7be2 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:21.368306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 563/1133] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:21.373541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c566653-c798-4b11-bde9-46841f7989e7 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:21.373978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c566653-c798-4b11-bde9-46841f7989e7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:21.374082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c566653-c798-4b11-bde9-46841f7989e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:21.441095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-7c566653-c798-4b11-bde9-46841f7989e7 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:21.442300 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7c566653-c798-4b11-bde9-46841f7989e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:21.471868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:21.538674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 571/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:21.831784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff8784-a668-4db8-b0de-c067f01e7be2 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:21.833359 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f6ff8784-a668-4db8-b0de-c067f01e7be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:21.935817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff8784-a668-4db8-b0de-c067f01e7be2 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:21.963257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:22.009330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ad40e-31eb-4313-9dc0-6094cf1e63e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:22.009854 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ad40e-31eb-4313-9dc0-6094cf1e63e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:22.010134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ad40e-31eb-4313-9dc0-6094cf1e63e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:22.010407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ad40e-31eb-4313-9dc0-6094cf1e63e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:22.012050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-158ad40e-31eb-4313-9dc0-6094cf1e63e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:22.120706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ad40e-31eb-4313-9dc0-6094cf1e63e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:22.122124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ad40e-31eb-4313-9dc0-6094cf1e63e3 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:22.140904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 564/1135] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:24.324540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92c57b-d07e-4d88-be68-cc3eecb5c90e None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:24.325416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92c57b-d07e-4d88-be68-cc3eecb5c90e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:24.325698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92c57b-d07e-4d88-be68-cc3eecb5c90e None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:24.327112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 572/1136] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:24.332295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6cd1-41fd-424b-b78d-4476424af831 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:24.332686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6cd1-41fd-424b-b78d-4476424af831 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:24.332836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6cd1-41fd-424b-b78d-4476424af831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:24.398251 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-539f6cd1-41fd-424b-b78d-4476424af831 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:24.399484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-539f6cd1-41fd-424b-b78d-4476424af831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:24.408213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:24.498589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 565/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:24.786481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92c57b-d07e-4d88-be68-cc3eecb5c90e None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:24.788045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0f92c57b-d07e-4d88-be68-cc3eecb5c90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:24.889559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92c57b-d07e-4d88-be68-cc3eecb5c90e None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:24.944192 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:24.962796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e02f71-5a70-4252-a9f4-f7b57706863c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:24.963439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e02f71-5a70-4252-a9f4-f7b57706863c service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:24.963720 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e02f71-5a70-4252-a9f4-f7b57706863c service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:24.963993 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e02f71-5a70-4252-a9f4-f7b57706863c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:24.965600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-73e02f71-5a70-4252-a9f4-f7b57706863c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:25.072938 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e02f71-5a70-4252-a9f4-f7b57706863c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:25.074358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e02f71-5a70-4252-a9f4-f7b57706863c service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:25.092488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 573/1138] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:27.279267 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229c7ef-ecfb-4fac-b347-8b47f2fbc0f4 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:27.279950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229c7ef-ecfb-4fac-b347-8b47f2fbc0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:27.279950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229c7ef-ecfb-4fac-b347-8b47f2fbc0f4 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:27.281216 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 566/1139] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:27.286493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d32c54-0b51-422b-bdd6-f871378e66d6 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:27.286925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d32c54-0b51-422b-bdd6-f871378e66d6 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:27.287063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d32c54-0b51-422b-bdd6-f871378e66d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:27.352913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-46d32c54-0b51-422b-bdd6-f871378e66d6 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:27.354137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-46d32c54-0b51-422b-bdd6-f871378e66d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:27.392687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:27.450403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 574/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:27.739724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229c7ef-ecfb-4fac-b347-8b47f2fbc0f4 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:27.741272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f229c7ef-ecfb-4fac-b347-8b47f2fbc0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:27.844526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229c7ef-ecfb-4fac-b347-8b47f2fbc0f4 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:27.882080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:27.917937 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751614f-c193-4653-addf-61cc580100ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:27.918537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751614f-c193-4653-addf-61cc580100ef service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:27.918815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751614f-c193-4653-addf-61cc580100ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:27.919114 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751614f-c193-4653-addf-61cc580100ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:27.921276 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8751614f-c193-4653-addf-61cc580100ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:28.036076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751614f-c193-4653-addf-61cc580100ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:28.037509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751614f-c193-4653-addf-61cc580100ef service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:28.055966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 567/1141] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:30.274868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05154868-4bd3-470d-bb34-cf95049417b7 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:30.275490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05154868-4bd3-470d-bb34-cf95049417b7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:30.275490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05154868-4bd3-470d-bb34-cf95049417b7 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:30.276884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 575/1142] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:30.282237 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f1ef4-7af3-4222-9809-b733376c08be None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:30.282624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f1ef4-7af3-4222-9809-b733376c08be None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:30.282792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f1ef4-7af3-4222-9809-b733376c08be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:30.346849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-430f1ef4-7af3-4222-9809-b733376c08be None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:30.348065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-430f1ef4-7af3-4222-9809-b733376c08be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:30.365710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:30.445475 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 568/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:30.733944 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05154868-4bd3-470d-bb34-cf95049417b7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:30.735548 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-05154868-4bd3-470d-bb34-cf95049417b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:30.842246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05154868-4bd3-470d-bb34-cf95049417b7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:30.907888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:30.914728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827158f3-6cf5-4643-aba5-af6a926828fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:30.915366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827158f3-6cf5-4643-aba5-af6a926828fd service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:30.915647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827158f3-6cf5-4643-aba5-af6a926828fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:30.915917 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827158f3-6cf5-4643-aba5-af6a926828fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:30.917513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-827158f3-6cf5-4643-aba5-af6a926828fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:31.025023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827158f3-6cf5-4643-aba5-af6a926828fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:31.026446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827158f3-6cf5-4643-aba5-af6a926828fd service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:31.045451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 576/1144] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:33.198884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c391fb1-eb91-4aa7-a6a4-f4779dbc918c None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:33.199504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c391fb1-eb91-4aa7-a6a4-f4779dbc918c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:33.199504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c391fb1-eb91-4aa7-a6a4-f4779dbc918c None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:33.201564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 569/1145] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:33.206871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de51f6-21a3-4712-b63a-c7ef06914988 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:33.207320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de51f6-21a3-4712-b63a-c7ef06914988 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:33.207437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de51f6-21a3-4712-b63a-c7ef06914988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:33.274579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f9de51f6-21a3-4712-b63a-c7ef06914988 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:33.275816 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f9de51f6-21a3-4712-b63a-c7ef06914988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:33.326188 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:33.375372 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 577/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:33.659738 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c391fb1-eb91-4aa7-a6a4-f4779dbc918c None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:33.661284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1c391fb1-eb91-4aa7-a6a4-f4779dbc918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:33.762876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c391fb1-eb91-4aa7-a6a4-f4779dbc918c None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:33.810835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:33.836660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc7d2-4b11-4646-b2fc-db391689fd53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:33.837262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc7d2-4b11-4646-b2fc-db391689fd53 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:33.837544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc7d2-4b11-4646-b2fc-db391689fd53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:33.837811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc7d2-4b11-4646-b2fc-db391689fd53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:33.839443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5f1dc7d2-4b11-4646-b2fc-db391689fd53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:33.946660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dc7d2-4b11-4646-b2fc-db391689fd53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:33.948105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dc7d2-4b11-4646-b2fc-db391689fd53 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:33.965813 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 570/1147] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:43.373824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=22 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:43.963480 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=66 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:53.881451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6316f4-56bd-4292-b9f8-5931615712c2 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:53.882020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6316f4-56bd-4292-b9f8-5931615712c2 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:53.882020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6316f4-56bd-4292-b9f8-5931615712c2 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:53.883376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 578/1148] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 19:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:53.888546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47f4e-9a32-4f95-ae64-5b8cb4a6087f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:53.888988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47f4e-9a32-4f95-ae64-5b8cb4a6087f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:53.889099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47f4e-9a32-4f95-ae64-5b8cb4a6087f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:53.955591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-aac47f4e-9a32-4f95-ae64-5b8cb4a6087f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:53.956791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-aac47f4e-9a32-4f95-ae64-5b8cb4a6087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:54.052746 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 571/1149] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 19:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:54.345669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1597113-8331-4f1f-a44b-5d3efeae1ec7 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:54.345949 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1597113-8331-4f1f-a44b-5d3efeae1ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:54.346221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1597113-8331-4f1f-a44b-5d3efeae1ec7 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:54.347569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 579/1150] 10.222.0.58 () {54 vars in 962 bytes} [Fri Oct 28 19:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:54.352437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69997c66-b621-410f-bbb4-46362b9b013b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:54.352722 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69997c66-b621-410f-bbb4-46362b9b013b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:54.352990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69997c66-b621-410f-bbb4-46362b9b013b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:54.428027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-69997c66-b621-410f-bbb4-46362b9b013b None None] MFA Rules not processed for user `6e8ec16bd9254b23bfd2b60551af9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:54.429197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-69997c66-b621-410f-bbb4-46362b9b013b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:54.448803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:54.507107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 572/1151] 10.222.0.58 () {58 vars in 1059 bytes} [Fri Oct 28 19:00:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:54.515528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1597113-8331-4f1f-a44b-5d3efeae1ec7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:54.517310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c1597113-8331-4f1f-a44b-5d3efeae1ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:54.619988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1597113-8331-4f1f-a44b-5d3efeae1ec7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:54.691906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7e533-4f84-4bd6-8c38-75451e210f7d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8ec16bd9254b23bfd2b60551af9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:54.692506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7e533-4f84-4bd6-8c38-75451e210f7d service glance] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:54.692780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7e533-4f84-4bd6-8c38-75451e210f7d service glance] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:54.693050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7e533-4f84-4bd6-8c38-75451e210f7d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:54.694643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9ec7e533-4f84-4bd6-8c38-75451e210f7d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:54.748458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:54.803065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7e533-4f84-4bd6-8c38-75451e210f7d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:54.804496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7e533-4f84-4bd6-8c38-75451e210f7d service glance] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:54.822885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 580/1152] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:00:57.038620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d9c66-85b3-44bc-9e3f-701f18c21220 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:57.039211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d9c66-85b3-44bc-9e3f-701f18c21220 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:57.039211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d9c66-85b3-44bc-9e3f-701f18c21220 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:57.040556 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 573/1153] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 19:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:57.045837 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b5c10-5fdb-4cd2-9209-df839a7e8bef None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:57.046257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b5c10-5fdb-4cd2-9209-df839a7e8bef None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:57.046364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b5c10-5fdb-4cd2-9209-df839a7e8bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:57.110412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-322b5c10-5fdb-4cd2-9209-df839a7e8bef None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:57.111618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-322b5c10-5fdb-4cd2-9209-df839a7e8bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:57.206316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 581/1154] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 19:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:00:57.479784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d9c66-85b3-44bc-9e3f-701f18c21220 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:00:57.481903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f43d9c66-85b3-44bc-9e3f-701f18c21220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:00:57.604134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d9c66-85b3-44bc-9e3f-701f18c21220 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:00:57.654675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:00:57.698835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808583c-5f38-429e-84b2-2576eeb45558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:00:57.699477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808583c-5f38-429e-84b2-2576eeb45558 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:57.699749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808583c-5f38-429e-84b2-2576eeb45558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:57.700013 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808583c-5f38-429e-84b2-2576eeb45558 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:57.700636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808583c-5f38-429e-84b2-2576eeb45558 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:00:57.701720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING py.warnings [None req-f808583c-5f38-429e-84b2-2576eeb45558 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:00:57.701720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: warnings.warn(msg) Oct 28 19:00:57.701720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: Oct 28 19:00:57.702851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808583c-5f38-429e-84b2-2576eeb45558 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:00:57.721581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 574/1155] 10.222.0.58 () {56 vars in 1108 bytes} [Fri Oct 28 19:00:57 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:00:59.889468 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acb87e-a2f7-4b13-9108-0ec17ef2441a None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:59.890044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acb87e-a2f7-4b13-9108-0ec17ef2441a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:59.890044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acb87e-a2f7-4b13-9108-0ec17ef2441a None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:59.891628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 582/1156] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 19:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:00:59.897244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c9ab2-6e56-4df5-b2eb-7407431ebdd5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:00:59.897644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c9ab2-6e56-4df5-b2eb-7407431ebdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:00:59.897778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c9ab2-6e56-4df5-b2eb-7407431ebdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:00:59.962510 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-8d5c9ab2-6e56-4df5-b2eb-7407431ebdd5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:00:59.963779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8d5c9ab2-6e56-4df5-b2eb-7407431ebdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:00.058376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 575/1157] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 19:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:01:00.352456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acb87e-a2f7-4b13-9108-0ec17ef2441a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:01:00.354009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b7acb87e-a2f7-4b13-9108-0ec17ef2441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:00.363954 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:00.497735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acb87e-a2f7-4b13-9108-0ec17ef2441a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:01:00.576048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:00.608660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51f105-7955-4bcb-a279-0e8e22899b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:01:00.609198 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51f105-7955-4bcb-a279-0e8e22899b47 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:00.609471 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51f105-7955-4bcb-a279-0e8e22899b47 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:00.609738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51f105-7955-4bcb-a279-0e8e22899b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:00.611344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5c51f105-7955-4bcb-a279-0e8e22899b47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:00.698399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51f105-7955-4bcb-a279-0e8e22899b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:01:00.700020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51f105-7955-4bcb-a279-0e8e22899b47 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:01:00.717262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 583/1158] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:01:03.047049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4431e-7ffa-40be-9ee7-beb86d91ce15 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:03.047595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4431e-7ffa-40be-9ee7-beb86d91ce15 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:03.047595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4431e-7ffa-40be-9ee7-beb86d91ce15 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:03.048929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 576/1159] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 19:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:01:03.054114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeda126-bcff-4aae-a0d7-eb3737d98630 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:03.054523 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeda126-bcff-4aae-a0d7-eb3737d98630 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:03.054661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeda126-bcff-4aae-a0d7-eb3737d98630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:03.119385 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8eeda126-bcff-4aae-a0d7-eb3737d98630 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:01:03.120572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8eeda126-bcff-4aae-a0d7-eb3737d98630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:03.166568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:03.217223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 584/1160] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 19:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:01:03.506159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4431e-7ffa-40be-9ee7-beb86d91ce15 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:01:03.507764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3be4431e-7ffa-40be-9ee7-beb86d91ce15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:03.532587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:03.610196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4431e-7ffa-40be-9ee7-beb86d91ce15 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:01:03.681734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b7d8c-003e-4b54-afc9-0ad783427ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:01:03.682323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b7d8c-003e-4b54-afc9-0ad783427ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:03.682595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b7d8c-003e-4b54-afc9-0ad783427ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:03.682856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b7d8c-003e-4b54-afc9-0ad783427ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:03.684560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-281b7d8c-003e-4b54-afc9-0ad783427ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:03.758230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:03.793522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b7d8c-003e-4b54-afc9-0ad783427ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:01:03.794956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b7d8c-003e-4b54-afc9-0ad783427ca7 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:01:03.813371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 577/1161] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:01:05.993833 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498956-332a-49eb-91cc-67027d423a0d None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:05.994371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498956-332a-49eb-91cc-67027d423a0d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:05.994371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498956-332a-49eb-91cc-67027d423a0d None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:05.995740 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 585/1162] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 19:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:01:06.000992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab91152-1504-4d83-bbe8-a4215b544c30 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:06.001402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab91152-1504-4d83-bbe8-a4215b544c30 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:06.001612 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab91152-1504-4d83-bbe8-a4215b544c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:06.066323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-5ab91152-1504-4d83-bbe8-a4215b544c30 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:01:06.067546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5ab91152-1504-4d83-bbe8-a4215b544c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:06.161097 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 578/1163] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 19:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:01:06.446305 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f498956-332a-49eb-91cc-67027d423a0d None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:01:06.447857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4f498956-332a-49eb-91cc-67027d423a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:06.550236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f498956-332a-49eb-91cc-67027d423a0d None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:01:06.621390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:06.622832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c75235-8e5e-4ec6-bc30-f19c4f7981b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:01:06.623639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c75235-8e5e-4ec6-bc30-f19c4f7981b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:06.624091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c75235-8e5e-4ec6-bc30-f19c4f7981b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:06.624512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c75235-8e5e-4ec6-bc30-f19c4f7981b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:06.626286 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-58c75235-8e5e-4ec6-bc30-f19c4f7981b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:06.734728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c75235-8e5e-4ec6-bc30-f19c4f7981b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:01:06.736117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c75235-8e5e-4ec6-bc30-f19c4f7981b4 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:01:06.754679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 586/1164] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:01:08.984675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b0903-80f9-4823-872d-b74958ef4c66 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:08.985270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b0903-80f9-4823-872d-b74958ef4c66 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:08.985270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b0903-80f9-4823-872d-b74958ef4c66 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:08.986535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 579/1165] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 19:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 19:01:08.991638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf43d-e7e1-4037-a217-2b1169a1d467 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:08.992100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf43d-e7e1-4037-a217-2b1169a1d467 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:08.992209 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf43d-e7e1-4037-a217-2b1169a1d467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:09.058261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-62ccf43d-e7e1-4037-a217-2b1169a1d467 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:01:09.059493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-62ccf43d-e7e1-4037-a217-2b1169a1d467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:09.110595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:09.154781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 587/1166] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 19:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:01:09.444002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b0903-80f9-4823-872d-b74958ef4c66 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:01:09.445549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6e3b0903-80f9-4823-872d-b74958ef4c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:09.482996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:09.549733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b0903-80f9-4823-872d-b74958ef4c66 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:01:09.620732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27b9b9-f362-484a-ad09-4ef41d2007a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:01:09.621529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27b9b9-f362-484a-ad09-4ef41d2007a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:09.621939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27b9b9-f362-484a-ad09-4ef41d2007a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:09.622348 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27b9b9-f362-484a-ad09-4ef41d2007a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:09.624096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0e27b9b9-f362-484a-ad09-4ef41d2007a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:09.704199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:09.735836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27b9b9-f362-484a-ad09-4ef41d2007a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:01:09.737264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27b9b9-f362-484a-ad09-4ef41d2007a8 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:01:09.755454 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 580/1167] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:01:12.062314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f2990-33cd-4769-812b-1bdbb703b127 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:12.062883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f2990-33cd-4769-812b-1bdbb703b127 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:12.062883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f2990-33cd-4769-812b-1bdbb703b127 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:12.064243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 588/1168] 10.222.0.58 () {54 vars in 927 bytes} [Fri Oct 28 19:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:01:12.069466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52a03d-3255-49b9-9c1d-58487c0e3c7d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:12.069865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52a03d-3255-49b9-9c1d-58487c0e3c7d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:12.070036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52a03d-3255-49b9-9c1d-58487c0e3c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:12.134599 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-8e52a03d-3255-49b9-9c1d-58487c0e3c7d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:01:12.135817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8e52a03d-3255-49b9-9c1d-58487c0e3c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:12.229759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 581/1169] 10.222.0.58 () {58 vars in 1024 bytes} [Fri Oct 28 19:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:01:12.522075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f2990-33cd-4769-812b-1bdbb703b127 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:01:12.523640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1f3f2990-33cd-4769-812b-1bdbb703b127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:12.625381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f2990-33cd-4769-812b-1bdbb703b127 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:01:12.696755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494acf6-8448-4a3d-99c1-2242d0711b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:01:12.697385 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494acf6-8448-4a3d-99c1-2242d0711b5d service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:01:12.697661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494acf6-8448-4a3d-99c1-2242d0711b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:01:12.697933 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494acf6-8448-4a3d-99c1-2242d0711b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:01:12.699544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7494acf6-8448-4a3d-99c1-2242d0711b5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:01:12.708691 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:12.808734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494acf6-8448-4a3d-99c1-2242d0711b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:01:12.810148 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494acf6-8448-4a3d-99c1-2242d0711b5d service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:01:12.828081 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 589/1170] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:01:22.228240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=82 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:01:22.825865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=50 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:25.556767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f4d67-922a-4198-955e-b8223be1baf4 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:25.557220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f4d67-922a-4198-955e-b8223be1baf4 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:25.557324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f4d67-922a-4198-955e-b8223be1baf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:25.630819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-1f4f4d67-922a-4198-955e-b8223be1baf4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:02:25.632036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1f4f4d67-922a-4198-955e-b8223be1baf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:25.725426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 582/1171] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:02:25.734003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bf577-c2b1-48ea-8335-ac7b1d2ebcb5 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:25.734494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bf577-c2b1-48ea-8335-ac7b1d2ebcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:25.734602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bf577-c2b1-48ea-8335-ac7b1d2ebcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:25.811379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-348bf577-c2b1-48ea-8335-ac7b1d2ebcb5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:02:25.812486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-348bf577-c2b1-48ea-8335-ac7b1d2ebcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:25.909330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 590/1172] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:02:25.913938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f4d67-922a-4198-955e-b8223be1baf4 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:25.915190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1f4f4d67-922a-4198-955e-b8223be1baf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:25.926697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f4d67-922a-4198-955e-b8223be1baf4 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:25.988033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:26.022749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:26.023961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:26.024236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:26.024502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:26.025134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:02:26.025439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:26.026621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING py.warnings [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:02:26.026621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: warnings.warn(msg) Oct 28 19:02:26.026621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: Oct 28 19:02:26.027548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:26.038427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 583/1173] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:02:26.047794 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bf577-c2b1-48ea-8335-ac7b1d2ebcb5 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:26.056934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bf577-c2b1-48ea-8335-ac7b1d2ebcb5 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:26.116903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:26.151143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:26.151765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:26.152042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:26.152315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:26.152948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:26.154039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING py.warnings [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:02:26.154039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: warnings.warn(msg) Oct 28 19:02:26.154039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: Oct 28 19:02:26.155263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:26.209577 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 591/1174] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:02:26 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:02:26.214297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:26.215834 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:26.247856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1bf30-4c42-4e00-98dc-887cac443162 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:26.341081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:26.341684 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:26.341961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:26.342233 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:26.342850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:26.343975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING py.warnings [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:02:26.343975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: warnings.warn(msg) Oct 28 19:02:26.343975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: Oct 28 19:02:26.345169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:26.387853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 584/1175] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:02:26 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:02:26.392143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:26.393632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:26.455255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6c1dc-ca15-4c5b-b66e-ac41f3870645 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:26.537219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:26.549129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed43ada-9564-4512-a0f0-6d223792ce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:26.549735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed43ada-9564-4512-a0f0-6d223792ce39 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:26.550013 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed43ada-9564-4512-a0f0-6d223792ce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:26.550286 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed43ada-9564-4512-a0f0-6d223792ce39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:26.550925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed43ada-9564-4512-a0f0-6d223792ce39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:26.552546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed43ada-9564-4512-a0f0-6d223792ce39 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:26.565224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 592/1176] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:02:26.569239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:26.571467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:26.582435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d15ce-d652-4e22-94db-bb425d6beb31 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:26.606424 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:26.676466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883fca8-6130-446d-b623-66e6381288b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:26.676987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883fca8-6130-446d-b623-66e6381288b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:26.677255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883fca8-6130-446d-b623-66e6381288b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:26.677518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883fca8-6130-446d-b623-66e6381288b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:26.678161 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883fca8-6130-446d-b623-66e6381288b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:26.680579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883fca8-6130-446d-b623-66e6381288b6 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:26.693165 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 585/1177] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:02:26.697279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed43ada-9564-4512-a0f0-6d223792ce39 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:26.698812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-aed43ada-9564-4512-a0f0-6d223792ce39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:26.732641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed43ada-9564-4512-a0f0-6d223792ce39 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:26.827740 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a293-5819-4304-8420-061a9e3853f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:26.828415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a293-5819-4304-8420-061a9e3853f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:26.828660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a293-5819-4304-8420-061a9e3853f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:26.828925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a293-5819-4304-8420-061a9e3853f0 admin admin] PATH_INFO: `/v3/projects/54251fd13d124acf923342a06622a313/users/d2cbfe4c370645b799661b0b28e2dc56/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:26.850728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1a293-5819-4304-8420-061a9e3853f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54251fd13d124acf923342a06622a313, user_id=d2cbfe4c370645b799661b0b28e2dc56, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:26.853992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1a293-5819-4304-8420-061a9e3853f0 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:26.878718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 593/1178] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:02:26 2022] PUT /identity/v3/projects/54251fd13d124acf923342a06622a313/users/d2cbfe4c370645b799661b0b28e2dc56/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:02:26.889941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0038ce-5680-4481-87ae-83073b3379cb None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:26.890223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0038ce-5680-4481-87ae-83073b3379cb None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:26.890496 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0038ce-5680-4481-87ae-83073b3379cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:26.946002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-5c0038ce-5680-4481-87ae-83073b3379cb None None] MFA Rules not processed for user `d2cbfe4c370645b799661b0b28e2dc56`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:02:26.947157 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5c0038ce-5680-4481-87ae-83073b3379cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:27.021594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 586/1179] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:02:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:02:27.034865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0a4ad-2e03-4b78-954a-024d7e311f73 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:27.035180 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0a4ad-2e03-4b78-954a-024d7e311f73 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:27.035460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0a4ad-2e03-4b78-954a-024d7e311f73 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:27.036831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 594/1180] 10.222.0.58 () {54 vars in 963 bytes} [Fri Oct 28 19:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:02:27.042045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47411243-6763-4587-aa53-089bc1f4031b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:27.042325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47411243-6763-4587-aa53-089bc1f4031b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:27.042594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47411243-6763-4587-aa53-089bc1f4031b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:27.073484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:27.118278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-47411243-6763-4587-aa53-089bc1f4031b None None] MFA Rules not processed for user `302ded4fdd72405c8ce8a364f7d2400c`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:02:27.119512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-47411243-6763-4587-aa53-089bc1f4031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:27.193184 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 587/1181] 10.222.0.58 () {58 vars in 1060 bytes} [Fri Oct 28 19:02:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:02:27.201001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0a4ad-2e03-4b78-954a-024d7e311f73 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:27.202451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5cf0a4ad-2e03-4b78-954a-024d7e311f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:27.252580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:27.305479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0a4ad-2e03-4b78-954a-024d7e311f73 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:27.377901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:27.378498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:27.378772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:27.379062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:27.380628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:27.477345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:27.478716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:27.482256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:27.496864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 595/1182] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:02:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:02:27.543177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd8a0b-6401-43c9-b2bf-75025533c2ff None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:27.543466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd8a0b-6401-43c9-b2bf-75025533c2ff None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:27.543740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd8a0b-6401-43c9-b2bf-75025533c2ff None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:27.544993 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 588/1183] 10.222.0.58 () {52 vars in 830 bytes} [Fri Oct 28 19:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:02:27.548633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:27.549842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:27.647651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b619c-a100-40af-9658-98049396f739 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:27.739991 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:27.740543 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:27.740822 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:27.741093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] PATH_INFO: `/v3/users/d2cbfe4c370645b799661b0b28e2dc56` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:27.754733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2cbfe4c370645b799661b0b28e2dc56)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:27.755860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING py.warnings [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:02:27.755860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: warnings.warn(msg) Oct 28 19:02:27.755860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: Oct 28 19:02:27.757093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:27.767773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:27.825927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2cbfe4c370645b799661b0b28e2dc56'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:27.826366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2cbfe4c370645b799661b0b28e2dc56'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:27.849389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2cbfe4c370645b799661b0b28e2dc56'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:27.863181 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2cbfe4c370645b799661b0b28e2dc56'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:27.863746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2cbfe4c370645b799661b0b28e2dc56'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:27.878376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 596/1184] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:02:27 2022] DELETE /identity/v3/users/d2cbfe4c370645b799661b0b28e2dc56 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:02:27.883419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2787d-94be-47ae-8c26-bc1cf5bc5a40 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:27.883922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2787d-94be-47ae-8c26-bc1cf5bc5a40 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:27.884021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2787d-94be-47ae-8c26-bc1cf5bc5a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:27.939504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-30f2787d-94be-47ae-8c26-bc1cf5bc5a40 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:02:27.940701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-30f2787d-94be-47ae-8c26-bc1cf5bc5a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:28.018697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:28.031211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 589/1185] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:02:28.041016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:28.042287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:28.141790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea9df3-cb4b-4f04-8d35-31674cb2dc62 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:28.195067 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:28.215844 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3f12d-86ed-45eb-bb03-244742f2bc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:28.216452 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3f12d-86ed-45eb-bb03-244742f2bc57 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:28.216731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3f12d-86ed-45eb-bb03-244742f2bc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:28.217001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3f12d-86ed-45eb-bb03-244742f2bc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:28.218597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5af3f12d-86ed-45eb-bb03-244742f2bc57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:28.326573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3f12d-86ed-45eb-bb03-244742f2bc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:28.327959 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3f12d-86ed-45eb-bb03-244742f2bc57 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:28.345787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 597/1186] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:02:28.672321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2787d-94be-47ae-8c26-bc1cf5bc5a40 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:02:28.674242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-30f2787d-94be-47ae-8c26-bc1cf5bc5a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:02:28.685579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2787d-94be-47ae-8c26-bc1cf5bc5a40 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:02:28.779862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:02:28.780491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:02:28.780767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:02:28.781033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] PATH_INFO: `/v3/projects/54251fd13d124acf923342a06622a313` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:02:28.791327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54251fd13d124acf923342a06622a313)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:02:28.792343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING py.warnings [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:02:28.792343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: warnings.warn(msg) Oct 28 19:02:28.792343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: Oct 28 19:02:28.793543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:02:28.849641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54251fd13d124acf923342a06622a313', 'request_id': 'req-6c6412bb-5ac2-49b8-9a59-5ba37552f364'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:28.863721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54251fd13d124acf923342a06622a313', 'request_id': 'req-6c6412bb-5ac2-49b8-9a59-5ba37552f364'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:28.872528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54251fd13d124acf923342a06622a313', 'request_id': 'req-6c6412bb-5ac2-49b8-9a59-5ba37552f364'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:28.897108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] The token cache is being invalidate because project 54251fd13d124acf923342a06622a313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:02:28.897753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:28.898673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c6412bb-5ac2-49b8-9a59-5ba37552f364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:02:28.900087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 590/1187] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:02:28 2022] DELETE /identity/v3/projects/54251fd13d124acf923342a06622a313 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:02:38.343529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=62 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:02:38.895140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:03:57.171787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55463e31-d983-4e1c-b555-491081d4a9b0 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:03:57.172399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55463e31-d983-4e1c-b555-491081d4a9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:03:57.172399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55463e31-d983-4e1c-b555-491081d4a9b0 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:03:57.173720 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 598/1188] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 19:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:03:57.179826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b9ed5-079f-4f54-a8ae-faf1db850985 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:03:57.180440 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b9ed5-079f-4f54-a8ae-faf1db850985 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:03:57.180440 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b9ed5-079f-4f54-a8ae-faf1db850985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:03:57.260857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6f1b9ed5-079f-4f54-a8ae-faf1db850985 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:03:57.262074 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6f1b9ed5-079f-4f54-a8ae-faf1db850985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:03:57.364937 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 591/1189] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 19:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:03:57.519902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55463e31-d983-4e1c-b555-491081d4a9b0 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:03:57.522660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-55463e31-d983-4e1c-b555-491081d4a9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:03:57.641290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55463e31-d983-4e1c-b555-491081d4a9b0 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:03:57.647368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a494a4a-e2c5-42dc-bf5c-0c68da529ed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:03:57.647973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a494a4a-e2c5-42dc-bf5c-0c68da529ed6 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:03:57.648256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a494a4a-e2c5-42dc-bf5c-0c68da529ed6 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:03:57.648526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a494a4a-e2c5-42dc-bf5c-0c68da529ed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:03:57.656469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a494a4a-e2c5-42dc-bf5c-0c68da529ed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:03:57.658054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a494a4a-e2c5-42dc-bf5c-0c68da529ed6 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:03:57.667315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 599/1190] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:03:58.492362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533211fd-ede6-40b3-a9a7-325b2965d941 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:03:58.492930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533211fd-ede6-40b3-a9a7-325b2965d941 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:03:58.492930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533211fd-ede6-40b3-a9a7-325b2965d941 None None] PATH_INFO: `/v3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:03:58.494873 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 592/1191] 10.222.0.45 () {54 vars in 925 bytes} [Fri Oct 28 19:03:58 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:03:58.500661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110571f-6b39-44f8-8b83-27be7e360530 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:03:58.501087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110571f-6b39-44f8-8b83-27be7e360530 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:03:58.501216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110571f-6b39-44f8-8b83-27be7e360530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:03:58.565877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-c110571f-6b39-44f8-8b83-27be7e360530 None None] MFA Rules not processed for user `da958657ac3041848d24a76b76705489`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:03:58.567189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c110571f-6b39-44f8-8b83-27be7e360530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:03:58.633187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:03:58.639231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 600/1192] 10.222.0.45 () {58 vars in 1015 bytes} [Fri Oct 28 19:03:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:03:58.741860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d740e06-6f3a-418d-962a-523a0cf79c06 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:03:58.742137 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d740e06-6f3a-418d-962a-523a0cf79c06 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:03:58.742489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d740e06-6f3a-418d-962a-523a0cf79c06 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:03:58.743833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 593/1193] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 19:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:03:58.748570 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7f05-fcb3-4153-a1af-d7a391f4c559 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:03:58.748859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7f05-fcb3-4153-a1af-d7a391f4c559 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:03:58.749135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7f05-fcb3-4153-a1af-d7a391f4c559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:03:58.769313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8d5c7f05-fcb3-4153-a1af-d7a391f4c559 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:03:58.770507 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8d5c7f05-fcb3-4153-a1af-d7a391f4c559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:03:58.778345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 601/1194] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 19:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:03:58.930536 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d740e06-6f3a-418d-962a-523a0cf79c06 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:03:58.944846 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d740e06-6f3a-418d-962a-523a0cf79c06 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:03:58.950343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7077a5-747b-405a-a295-0dc0de26815d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:03:58.950937 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7077a5-747b-405a-a295-0dc0de26815d service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:03:58.951239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7077a5-747b-405a-a295-0dc0de26815d service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:03:58.951507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7077a5-747b-405a-a295-0dc0de26815d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:03:58.959563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7077a5-747b-405a-a295-0dc0de26815d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:03:58.961195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7077a5-747b-405a-a295-0dc0de26815d service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:03:58.970417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 594/1195] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:04:07.292290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d1b4a-88a9-4d9b-b367-ff0f7eebb796 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:07.292780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d1b4a-88a9-4d9b-b367-ff0f7eebb796 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:07.292893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d1b4a-88a9-4d9b-b367-ff0f7eebb796 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:07.294144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 602/1196] 10.222.0.58 () {54 vars in 961 bytes} [Fri Oct 28 19:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:04:07.299236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a014-f61c-44d6-8bdc-669ceb61c017 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:07.299674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a014-f61c-44d6-8bdc-669ceb61c017 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:07.299776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a014-f61c-44d6-8bdc-669ceb61c017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:07.319533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3327a014-f61c-44d6-8bdc-669ceb61c017 None None] MFA Rules not processed for user `da958657ac3041848d24a76b76705489`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:04:07.320723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3327a014-f61c-44d6-8bdc-669ceb61c017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:07.329138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 595/1197] 10.222.0.58 () {58 vars in 1058 bytes} [Fri Oct 28 19:04:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:04:07.336957 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d1b4a-88a9-4d9b-b367-ff0f7eebb796 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:04:07.350380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d1b4a-88a9-4d9b-b367-ff0f7eebb796 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:04:07.355366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80e92b-c407-455e-a019-efba2064adc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da958657ac3041848d24a76b76705489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:04:07.356027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80e92b-c407-455e-a019-efba2064adc7 service placement] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:07.356368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80e92b-c407-455e-a019-efba2064adc7 service placement] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:07.356703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80e92b-c407-455e-a019-efba2064adc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:07.364931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80e92b-c407-455e-a019-efba2064adc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:04:07.366491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80e92b-c407-455e-a019-efba2064adc7 service placement] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:04:07.375820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 603/1198] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:04:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:04:10.417610 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7357c-79ea-4507-942e-7ab72d9f2d40 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:10.418180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7357c-79ea-4507-942e-7ab72d9f2d40 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:10.418180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7357c-79ea-4507-942e-7ab72d9f2d40 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:10.419543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 596/1199] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 19:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:04:10.424498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8156daa-6b3e-430d-af10-7c4f237dcbb6 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:10.424936 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8156daa-6b3e-430d-af10-7c4f237dcbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:10.425069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8156daa-6b3e-430d-af10-7c4f237dcbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:10.446637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f8156daa-6b3e-430d-af10-7c4f237dcbb6 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:04:10.448349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f8156daa-6b3e-430d-af10-7c4f237dcbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:10.457503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 604/1200] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 19:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:04:10.613069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7357c-79ea-4507-942e-7ab72d9f2d40 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:04:10.627857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7357c-79ea-4507-942e-7ab72d9f2d40 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:04:10.634072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dadc4-d679-4610-805a-1b726f1fbed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:04:10.634670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dadc4-d679-4610-805a-1b726f1fbed6 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:10.634947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dadc4-d679-4610-805a-1b726f1fbed6 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:10.635250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dadc4-d679-4610-805a-1b726f1fbed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:10.643606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dadc4-d679-4610-805a-1b726f1fbed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:04:10.645499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dadc4-d679-4610-805a-1b726f1fbed6 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:04:10.654903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 597/1201] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:04:11.698921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff50d82-8db6-46d8-8b5a-3f8189143936 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:11.699506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff50d82-8db6-46d8-8b5a-3f8189143936 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:11.699506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff50d82-8db6-46d8-8b5a-3f8189143936 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:11.700874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 605/1202] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 19:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:04:11.705479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63389f7a-c57b-4283-89ce-3f5037cfa377 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:11.705877 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63389f7a-c57b-4283-89ce-3f5037cfa377 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:11.706019 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63389f7a-c57b-4283-89ce-3f5037cfa377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:11.725385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-63389f7a-c57b-4283-89ce-3f5037cfa377 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:04:11.726560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-63389f7a-c57b-4283-89ce-3f5037cfa377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:11.734215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 598/1203] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 19:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:04:11.886613 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff50d82-8db6-46d8-8b5a-3f8189143936 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:04:11.901070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff50d82-8db6-46d8-8b5a-3f8189143936 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:04:11.906629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b57a0-b610-43c0-88b7-fcfb568eb865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:04:11.907461 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b57a0-b610-43c0-88b7-fcfb568eb865 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:11.907850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b57a0-b610-43c0-88b7-fcfb568eb865 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:11.908126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b57a0-b610-43c0-88b7-fcfb568eb865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:11.916687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b57a0-b610-43c0-88b7-fcfb568eb865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:04:11.918311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b57a0-b610-43c0-88b7-fcfb568eb865 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:04:11.928127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 606/1204] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:04:16.086121 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9844ca-cc97-4393-916d-cc64ba8fcffa None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:16.086696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9844ca-cc97-4393-916d-cc64ba8fcffa None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:16.086696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9844ca-cc97-4393-916d-cc64ba8fcffa None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:16.088064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 599/1205] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:04:16.093118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8875-64a0-465c-81d1-ac14a3c9999d None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:16.093550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8875-64a0-465c-81d1-ac14a3c9999d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:16.093662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8875-64a0-465c-81d1-ac14a3c9999d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:16.112991 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a8fc8875-64a0-465c-81d1-ac14a3c9999d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:04:16.114169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a8fc8875-64a0-465c-81d1-ac14a3c9999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:16.122458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 607/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:04:16.407726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9844ca-cc97-4393-916d-cc64ba8fcffa None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:04:16.409411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7a9844ca-cc97-4393-916d-cc64ba8fcffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:16.442646 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:04:16.488544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9844ca-cc97-4393-916d-cc64ba8fcffa None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:04:16.492255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939df3f8-c85a-4e89-9fdc-b0cf3db7920f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:04:16.492855 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939df3f8-c85a-4e89-9fdc-b0cf3db7920f service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:16.493133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939df3f8-c85a-4e89-9fdc-b0cf3db7920f service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:16.493403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939df3f8-c85a-4e89-9fdc-b0cf3db7920f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:16.502256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939df3f8-c85a-4e89-9fdc-b0cf3db7920f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:04:16.503712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939df3f8-c85a-4e89-9fdc-b0cf3db7920f service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:04:16.513255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 600/1207] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:04:18.678626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5da4e-c081-47af-b083-86a91d8835e4 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:18.679239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5da4e-c081-47af-b083-86a91d8835e4 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:18.679239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5da4e-c081-47af-b083-86a91d8835e4 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:18.680518 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 608/1208] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:04:18.685725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b99cd-db8f-4336-a4aa-6f0b6dfb6a76 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:18.686167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b99cd-db8f-4336-a4aa-6f0b6dfb6a76 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:18.686275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b99cd-db8f-4336-a4aa-6f0b6dfb6a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:18.705470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-411b99cd-db8f-4336-a4aa-6f0b6dfb6a76 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:04:18.706661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-411b99cd-db8f-4336-a4aa-6f0b6dfb6a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:18.714905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 601/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:04:19.002394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5da4e-c081-47af-b083-86a91d8835e4 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:04:19.004138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-50e5da4e-c081-47af-b083-86a91d8835e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:19.017795 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5da4e-c081-47af-b083-86a91d8835e4 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:04:19.021788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be437ee7-e939-488a-87fa-f42806716b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:04:19.022371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be437ee7-e939-488a-87fa-f42806716b1e service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:19.022646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be437ee7-e939-488a-87fa-f42806716b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:19.022913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be437ee7-e939-488a-87fa-f42806716b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:19.031528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be437ee7-e939-488a-87fa-f42806716b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:04:19.032937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be437ee7-e939-488a-87fa-f42806716b1e service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:04:19.042730 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 609/1210] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:04:22.708037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc557098-dd14-4383-80bc-d5f826e20d96 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:22.709113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc557098-dd14-4383-80bc-d5f826e20d96 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:22.709113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc557098-dd14-4383-80bc-d5f826e20d96 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:22.710485 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 602/1211] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:04:22.715631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3da0d-59de-47fd-9832-6e2411a7d1a2 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:22.716043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3da0d-59de-47fd-9832-6e2411a7d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:22.716178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3da0d-59de-47fd-9832-6e2411a7d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:22.735809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-14f3da0d-59de-47fd-9832-6e2411a7d1a2 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:04:22.736990 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-14f3da0d-59de-47fd-9832-6e2411a7d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:22.744561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 610/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:04:23.046819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc557098-dd14-4383-80bc-d5f826e20d96 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:04:23.049130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-dc557098-dd14-4383-80bc-d5f826e20d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:23.125800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc557098-dd14-4383-80bc-d5f826e20d96 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:04:23.129348 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b251a81-532d-413c-bd2e-1048ae31e16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:04:23.129950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b251a81-532d-413c-bd2e-1048ae31e16b service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:23.130766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b251a81-532d-413c-bd2e-1048ae31e16b service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:23.131064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b251a81-532d-413c-bd2e-1048ae31e16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:23.140028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b251a81-532d-413c-bd2e-1048ae31e16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:04:23.141461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b251a81-532d-413c-bd2e-1048ae31e16b service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:04:23.151553 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 603/1213] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:04:25.301888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd465d8-2f7b-4d84-81da-544690f033b7 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:25.302437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd465d8-2f7b-4d84-81da-544690f033b7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:25.302437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd465d8-2f7b-4d84-81da-544690f033b7 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:25.303827 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 611/1214] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 19:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:04:25.309126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9be40-b4a4-4367-8060-3c66c8b35271 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:25.309554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9be40-b4a4-4367-8060-3c66c8b35271 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:25.309708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9be40-b4a4-4367-8060-3c66c8b35271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:25.329573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4ce9be40-b4a4-4367-8060-3c66c8b35271 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:04:25.330782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4ce9be40-b4a4-4367-8060-3c66c8b35271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:25.339308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 604/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 19:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:04:25.636556 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd465d8-2f7b-4d84-81da-544690f033b7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:04:25.638231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6cd465d8-2f7b-4d84-81da-544690f033b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:04:25.651481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd465d8-2f7b-4d84-81da-544690f033b7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:04:25.654911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5dbc0-87d7-4a84-b549-b4b90664e14d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:04:25.655527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5dbc0-87d7-4a84-b549-b4b90664e14d service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:04:25.655803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5dbc0-87d7-4a84-b549-b4b90664e14d service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:04:25.656071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5dbc0-87d7-4a84-b549-b4b90664e14d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:04:25.664626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5dbc0-87d7-4a84-b549-b4b90664e14d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:04:25.666035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5dbc0-87d7-4a84-b549-b4b90664e14d service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:04:25.675430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 612/1216] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:04:35.324696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=63 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:04:35.671066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=37 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:48.447042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a86630-ec93-4f0e-877e-c30b0b8730de None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:48.448174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a86630-ec93-4f0e-877e-c30b0b8730de None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:48.448672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a86630-ec93-4f0e-877e-c30b0b8730de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:48.474715 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-58a86630-ec93-4f0e-877e-c30b0b8730de None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:48.480417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-58a86630-ec93-4f0e-877e-c30b0b8730de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:48.493317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 605/1217] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:48.503552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98298c90-d760-4e2a-8990-1181323c9520 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:48.504702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98298c90-d760-4e2a-8990-1181323c9520 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:48.505201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98298c90-d760-4e2a-8990-1181323c9520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:48.529068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-98298c90-d760-4e2a-8990-1181323c9520 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:48.530627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-98298c90-d760-4e2a-8990-1181323c9520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:48.543675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 613/1218] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:48.547892 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a86630-ec93-4f0e-877e-c30b0b8730de None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:48.560344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a86630-ec93-4f0e-877e-c30b0b8730de None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:48.565605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a958d5-7e5b-4ac7-8984-7496faa7647d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:48.566593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a958d5-7e5b-4ac7-8984-7496faa7647d admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:48.567174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a958d5-7e5b-4ac7-8984-7496faa7647d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:48.567686 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a958d5-7e5b-4ac7-8984-7496faa7647d admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:48.568550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a958d5-7e5b-4ac7-8984-7496faa7647d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:05:48.569076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a958d5-7e5b-4ac7-8984-7496faa7647d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:48.570912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a958d5-7e5b-4ac7-8984-7496faa7647d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:48.584640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 606/1219] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:05:48.592258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98298c90-d760-4e2a-8990-1181323c9520 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:48.604037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98298c90-d760-4e2a-8990-1181323c9520 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:48.613305 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aae0c5-94d3-4b6c-98a7-97ce39ba500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:48.614241 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aae0c5-94d3-4b6c-98a7-97ce39ba500f admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:48.614752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aae0c5-94d3-4b6c-98a7-97ce39ba500f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:48.615269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aae0c5-94d3-4b6c-98a7-97ce39ba500f admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:48.616141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aae0c5-94d3-4b6c-98a7-97ce39ba500f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:48.618277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aae0c5-94d3-4b6c-98a7-97ce39ba500f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:48.671082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 614/1220] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:48 2022] POST /identity/v3/projects => generated 389 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:48.675394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a958d5-7e5b-4ac7-8984-7496faa7647d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:48.686485 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a958d5-7e5b-4ac7-8984-7496faa7647d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:48.782580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdcde3-2f1a-435d-8827-88281cbaa142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:48.783660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdcde3-2f1a-435d-8827-88281cbaa142 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:48.784163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdcde3-2f1a-435d-8827-88281cbaa142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:48.784648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdcde3-2f1a-435d-8827-88281cbaa142 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:48.786077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdcde3-2f1a-435d-8827-88281cbaa142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:48.788467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdcde3-2f1a-435d-8827-88281cbaa142 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:48.825235 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 607/1221] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:48 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:48.829139 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aae0c5-94d3-4b6c-98a7-97ce39ba500f admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:48.840911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aae0c5-94d3-4b6c-98a7-97ce39ba500f admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:48.852103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f986d39-e225-46d3-871f-2d3ab7123dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:48.853039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f986d39-e225-46d3-871f-2d3ab7123dfe admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:48.853534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f986d39-e225-46d3-871f-2d3ab7123dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:48.854019 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f986d39-e225-46d3-871f-2d3ab7123dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:48.854878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f986d39-e225-46d3-871f-2d3ab7123dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:48.856786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f986d39-e225-46d3-871f-2d3ab7123dfe admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:48.871759 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 615/1222] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:48.875434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdcde3-2f1a-435d-8827-88281cbaa142 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:48.886126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdcde3-2f1a-435d-8827-88281cbaa142 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:48.895411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e25b39-c03e-42fb-b1ca-d3c9feddeafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:48.896313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e25b39-c03e-42fb-b1ca-d3c9feddeafd admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:48.896942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e25b39-c03e-42fb-b1ca-d3c9feddeafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:48.897439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e25b39-c03e-42fb-b1ca-d3c9feddeafd admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:48.898298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e25b39-c03e-42fb-b1ca-d3c9feddeafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:48.900145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e25b39-c03e-42fb-b1ca-d3c9feddeafd admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:48.913111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 608/1223] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:48.917006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f986d39-e225-46d3-871f-2d3ab7123dfe admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:48.928115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f986d39-e225-46d3-871f-2d3ab7123dfe admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:48.936046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b488f71-6558-41b7-ba89-fc473a727a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:48.936997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b488f71-6558-41b7-ba89-fc473a727a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:48.937508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b488f71-6558-41b7-ba89-fc473a727a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:48.938088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b488f71-6558-41b7-ba89-fc473a727a3d admin admin] PATH_INFO: `/v3/projects/55bc75ee717c43b7811b0ab11c647f99/users/e868bb74b33f467380562aec01a3d3e9/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:48.956118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b488f71-6558-41b7-ba89-fc473a727a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=55bc75ee717c43b7811b0ab11c647f99, user_id=e868bb74b33f467380562aec01a3d3e9, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:48.959304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b488f71-6558-41b7-ba89-fc473a727a3d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:48.986557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 616/1224] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:48 2022] PUT /identity/v3/projects/55bc75ee717c43b7811b0ab11c647f99/users/e868bb74b33f467380562aec01a3d3e9/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:48.992711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d841-f07a-4afb-bd7d-ef4e2f55b415 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:48.993427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d841-f07a-4afb-bd7d-ef4e2f55b415 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:48.993930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d841-f07a-4afb-bd7d-ef4e2f55b415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.020257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-9893d841-f07a-4afb-bd7d-ef4e2f55b415 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.022081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9893d841-f07a-4afb-bd7d-ef4e2f55b415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.115183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 609/1225] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.130627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b488f71-6558-41b7-ba89-fc473a727a3d admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.212555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd57ffe-c52f-4e5a-b442-98393cf812e5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.213309 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd57ffe-c52f-4e5a-b442-98393cf812e5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.213809 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd57ffe-c52f-4e5a-b442-98393cf812e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.233324 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b488f71-6558-41b7-ba89-fc473a727a3d admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.242191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6cd57ffe-c52f-4e5a-b442-98393cf812e5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.248214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0521e-0353-47e3-8aad-c4f8116123d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.249122 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0521e-0353-47e3-8aad-c4f8116123d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.249618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0521e-0353-47e3-8aad-c4f8116123d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.250116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0521e-0353-47e3-8aad-c4f8116123d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.253445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6cd57ffe-c52f-4e5a-b442-98393cf812e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.273129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0521e-0353-47e3-8aad-c4f8116123d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.276020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0521e-0353-47e3-8aad-c4f8116123d8 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.277754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 610/1226] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:05:49.291709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75aadf-7e89-431d-adef-8a0b3db6a810 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.292347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75aadf-7e89-431d-adef-8a0b3db6a810 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.292837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75aadf-7e89-431d-adef-8a0b3db6a810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.302791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 617/1227] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:05:49.320396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4d75aadf-7e89-431d-adef-8a0b3db6a810 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.321750 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4d75aadf-7e89-431d-adef-8a0b3db6a810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.334293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 611/1228] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:05:49.348932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0521e-0353-47e3-8aad-c4f8116123d8 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.360915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0521e-0353-47e3-8aad-c4f8116123d8 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.375006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a83c02-3b84-497a-92f3-653dba4403e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.375006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a83c02-3b84-497a-92f3-653dba4403e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.375006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a83c02-3b84-497a-92f3-653dba4403e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.375006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a83c02-3b84-497a-92f3-653dba4403e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.375454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a83c02-3b84-497a-92f3-653dba4403e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:05:49.375454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a83c02-3b84-497a-92f3-653dba4403e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.375454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING py.warnings [None req-04a83c02-3b84-497a-92f3-653dba4403e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:05:49.375454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: warnings.warn(msg) Oct 28 19:05:49.375454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: Oct 28 19:05:49.375454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a83c02-3b84-497a-92f3-653dba4403e7 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.402435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 618/1229] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:05:49.436826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75aadf-7e89-431d-adef-8a0b3db6a810 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.451977 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75aadf-7e89-431d-adef-8a0b3db6a810 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.452912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226d8f5-7eee-432b-93f3-900313f1e29f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.453730 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226d8f5-7eee-432b-93f3-900313f1e29f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.454159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226d8f5-7eee-432b-93f3-900313f1e29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.465032 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b395f-b720-4a89-929d-a7f3601a0f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.472326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b395f-b720-4a89-929d-a7f3601a0f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.472780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b395f-b720-4a89-929d-a7f3601a0f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.478174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b395f-b720-4a89-929d-a7f3601a0f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.478174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b395f-b720-4a89-929d-a7f3601a0f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.478174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING py.warnings [None req-c77b395f-b720-4a89-929d-a7f3601a0f09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:05:49.478174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: warnings.warn(msg) Oct 28 19:05:49.478174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: Oct 28 19:05:49.479973 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b395f-b720-4a89-929d-a7f3601a0f09 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.506966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-d226d8f5-7eee-432b-93f3-900313f1e29f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.509039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d226d8f5-7eee-432b-93f3-900313f1e29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.520252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 619/1230] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.520952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463ed3d-a25e-4e5e-9cfe-36d6daaa7a56 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.521235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463ed3d-a25e-4e5e-9cfe-36d6daaa7a56 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.521549 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463ed3d-a25e-4e5e-9cfe-36d6daaa7a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.533280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 612/1231] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/projects => generated 357 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:49.533280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a87c-0b81-4c5f-924f-654742aea6e5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.533280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a87c-0b81-4c5f-924f-654742aea6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.533280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a87c-0b81-4c5f-924f-654742aea6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.548671 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f463ed3d-a25e-4e5e-9cfe-36d6daaa7a56 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.549983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f463ed3d-a25e-4e5e-9cfe-36d6daaa7a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.555503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-9162a87c-0b81-4c5f-924f-654742aea6e5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.557899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9162a87c-0b81-4c5f-924f-654742aea6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.586296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:49.623554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 620/1232] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.624902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9adba3-0268-4196-a047-40f45bbf829b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.625191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9adba3-0268-4196-a047-40f45bbf829b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.625465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9adba3-0268-4196-a047-40f45bbf829b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.647111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 613/1233] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.648852 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d139c3-e01d-4e9d-92fb-0cf2b01773be None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.649156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d139c3-e01d-4e9d-92fb-0cf2b01773be None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.650015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d139c3-e01d-4e9d-92fb-0cf2b01773be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.651542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ae9adba3-0268-4196-a047-40f45bbf829b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.652920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ae9adba3-0268-4196-a047-40f45bbf829b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.671034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 621/1234] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.671034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350ff06-2794-4479-bd6f-680f88cd648f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.671034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350ff06-2794-4479-bd6f-680f88cd648f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.671034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350ff06-2794-4479-bd6f-680f88cd648f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.679695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:49.684679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d6d139c3-e01d-4e9d-92fb-0cf2b01773be None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.685878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d6d139c3-e01d-4e9d-92fb-0cf2b01773be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.694164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-9350ff06-2794-4479-bd6f-680f88cd648f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.703196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 614/1235] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.703196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d139c3-e01d-4e9d-92fb-0cf2b01773be None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.703437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9350ff06-2794-4479-bd6f-680f88cd648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.705029 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d139c3-e01d-4e9d-92fb-0cf2b01773be None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.710290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8bca6-cd2c-46a0-86f7-37be14ca5b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.710891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8bca6-cd2c-46a0-86f7-37be14ca5b78 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.711462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8bca6-cd2c-46a0-86f7-37be14ca5b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.711585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 622/1236] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.712595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f34aff-e95c-4207-b3ea-48ab675affae None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.712884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f34aff-e95c-4207-b3ea-48ab675affae None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.713153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f34aff-e95c-4207-b3ea-48ab675affae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.715281 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8bca6-cd2c-46a0-86f7-37be14ca5b78 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.716567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8bca6-cd2c-46a0-86f7-37be14ca5b78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.719266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8bca6-cd2c-46a0-86f7-37be14ca5b78 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.758032 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 615/1237] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/users => generated 279 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:49.758956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1657c1-3bc3-477e-8519-1cd616691134 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.759630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1657c1-3bc3-477e-8519-1cd616691134 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.759630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1657c1-3bc3-477e-8519-1cd616691134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.766145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-c2f34aff-e95c-4207-b3ea-48ab675affae None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.767576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c2f34aff-e95c-4207-b3ea-48ab675affae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.782373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3f1657c1-3bc3-477e-8519-1cd616691134 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.783716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3f1657c1-3bc3-477e-8519-1cd616691134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.785096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 623/1238] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.786862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a325-8cea-4586-94c6-3ea96c0ec379 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.787386 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a325-8cea-4586-94c6-3ea96c0ec379 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.787942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a325-8cea-4586-94c6-3ea96c0ec379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.795848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 616/1239] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.795848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03746dc4-0aeb-4b19-a670-07b978fb2d0d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.795848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03746dc4-0aeb-4b19-a670-07b978fb2d0d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.795848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03746dc4-0aeb-4b19-a670-07b978fb2d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.817975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-80c4a325-8cea-4586-94c6-3ea96c0ec379 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.818672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-03746dc4-0aeb-4b19-a670-07b978fb2d0d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.819238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-80c4a325-8cea-4586-94c6-3ea96c0ec379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.820005 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-03746dc4-0aeb-4b19-a670-07b978fb2d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.827449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 624/1240] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.828396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 617/1241] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.829661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ce717-949b-4183-af99-321001cff129 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.830018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ce717-949b-4183-af99-321001cff129 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.830352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ce717-949b-4183-af99-321001cff129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.833333 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461152b-3581-4d7e-a2d8-74f975e1c409 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.833616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461152b-3581-4d7e-a2d8-74f975e1c409 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.834468 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461152b-3581-4d7e-a2d8-74f975e1c409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.852443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-165ce717-949b-4183-af99-321001cff129 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.853848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-165ce717-949b-4183-af99-321001cff129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.855135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-5461152b-3581-4d7e-a2d8-74f975e1c409 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:49.856348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5461152b-3581-4d7e-a2d8-74f975e1c409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:49.863064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 618/1242] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.864122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ce717-949b-4183-af99-321001cff129 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.868378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 625/1243] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:49.868378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461152b-3581-4d7e-a2d8-74f975e1c409 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.877955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ce717-949b-4183-af99-321001cff129 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.881020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461152b-3581-4d7e-a2d8-74f975e1c409 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.883474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec2ccf-3cb7-42cc-8625-bb5406f66542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.884509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec2ccf-3cb7-42cc-8625-bb5406f66542 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.884808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec2ccf-3cb7-42cc-8625-bb5406f66542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.885015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec2ccf-3cb7-42cc-8625-bb5406f66542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.885655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec2ccf-3cb7-42cc-8625-bb5406f66542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.887274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec2ccf-3cb7-42cc-8625-bb5406f66542 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.887879 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730962cd-d525-49a7-aac4-3500225b7b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.888584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730962cd-d525-49a7-aac4-3500225b7b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.889039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730962cd-d525-49a7-aac4-3500225b7b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.889405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730962cd-d525-49a7-aac4-3500225b7b36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.890170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730962cd-d525-49a7-aac4-3500225b7b36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:05:49.890589 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730962cd-d525-49a7-aac4-3500225b7b36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.892274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730962cd-d525-49a7-aac4-3500225b7b36 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.901911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 619/1244] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:49.901911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec2ccf-3cb7-42cc-8625-bb5406f66542 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.912965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 626/1245] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:05:49.912965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730962cd-d525-49a7-aac4-3500225b7b36 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.913304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec2ccf-3cb7-42cc-8625-bb5406f66542 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.917971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0882bb2-cb38-49a2-8acd-7d29c8045570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.918864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0882bb2-cb38-49a2-8acd-7d29c8045570 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.919288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0882bb2-cb38-49a2-8acd-7d29c8045570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.919681 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0882bb2-cb38-49a2-8acd-7d29c8045570 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.920310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0882bb2-cb38-49a2-8acd-7d29c8045570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:05:49.920578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0882bb2-cb38-49a2-8acd-7d29c8045570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.922201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0882bb2-cb38-49a2-8acd-7d29c8045570 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.925696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730962cd-d525-49a7-aac4-3500225b7b36 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.930536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeecad1-f4c6-42c4-9900-c2c1246ad7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.931295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeecad1-f4c6-42c4-9900-c2c1246ad7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.931761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeecad1-f4c6-42c4-9900-c2c1246ad7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.932060 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeecad1-f4c6-42c4-9900-c2c1246ad7c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.932904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeecad1-f4c6-42c4-9900-c2c1246ad7c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:05:49.933463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeecad1-f4c6-42c4-9900-c2c1246ad7c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.935274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeecad1-f4c6-42c4-9900-c2c1246ad7c6 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.936428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 620/1246] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:05:49.945295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0882bb2-cb38-49a2-8acd-7d29c8045570 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.958046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 627/1247] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:05:49.958046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeecad1-f4c6-42c4-9900-c2c1246ad7c6 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.958337 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0882bb2-cb38-49a2-8acd-7d29c8045570 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.963160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649086c8-99fe-42c7-b9ca-55821f6ccc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.963766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649086c8-99fe-42c7-b9ca-55821f6ccc32 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.964673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649086c8-99fe-42c7-b9ca-55821f6ccc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.964849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649086c8-99fe-42c7-b9ca-55821f6ccc32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.965731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649086c8-99fe-42c7-b9ca-55821f6ccc32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:05:49.966290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649086c8-99fe-42c7-b9ca-55821f6ccc32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.968198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649086c8-99fe-42c7-b9ca-55821f6ccc32 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.968401 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeecad1-f4c6-42c4-9900-c2c1246ad7c6 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:49.974701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5b004-1da2-4948-ab4b-e14d929270f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:49.975350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5b004-1da2-4948-ab4b-e14d929270f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:49.975627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5b004-1da2-4948-ab4b-e14d929270f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:49.975957 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5b004-1da2-4948-ab4b-e14d929270f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:49.976533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5b004-1da2-4948-ab4b-e14d929270f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:05:49.977806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5b004-1da2-4948-ab4b-e14d929270f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:49.978418 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5b004-1da2-4948-ab4b-e14d929270f7 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:49.997685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 628/1248] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:05:49.997897 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 621/1249] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:05:49.997999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5b004-1da2-4948-ab4b-e14d929270f7 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:49.998107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649086c8-99fe-42c7-b9ca-55821f6ccc32 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.008360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5b004-1da2-4948-ab4b-e14d929270f7 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.009874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649086c8-99fe-42c7-b9ca-55821f6ccc32 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.013380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0ca75-51fc-4c22-b805-cefd072bbf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.014055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0ca75-51fc-4c22-b805-cefd072bbf29 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.014405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0ca75-51fc-4c22-b805-cefd072bbf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.014738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0ca75-51fc-4c22-b805-cefd072bbf29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.015646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0ca75-51fc-4c22-b805-cefd072bbf29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:05:50.015802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a6d6-6221-4416-89df-78edad1e3b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.015927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0ca75-51fc-4c22-b805-cefd072bbf29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.016642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a6d6-6221-4416-89df-78edad1e3b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.016642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a6d6-6221-4416-89df-78edad1e3b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.017174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a6d6-6221-4416-89df-78edad1e3b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.017955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0ca75-51fc-4c22-b805-cefd072bbf29 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.018104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164a6d6-6221-4416-89df-78edad1e3b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.019647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164a6d6-6221-4416-89df-78edad1e3b7d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.032134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 622/1250] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:50.032634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 629/1251] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:05:50.033074 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a6d6-6221-4416-89df-78edad1e3b7d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.033427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0ca75-51fc-4c22-b805-cefd072bbf29 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.044090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a6d6-6221-4416-89df-78edad1e3b7d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.044280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0ca75-51fc-4c22-b805-cefd072bbf29 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.049061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6c4da-631a-4c06-b633-fb89564a3210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.049413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3c9d8-f5c9-46ef-9844-2a31cb4df87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.049872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6c4da-631a-4c06-b633-fb89564a3210 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.050255 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3c9d8-f5c9-46ef-9844-2a31cb4df87a admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.050371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6c4da-631a-4c06-b633-fb89564a3210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.050693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3c9d8-f5c9-46ef-9844-2a31cb4df87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.050818 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6c4da-631a-4c06-b633-fb89564a3210 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.051240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3c9d8-f5c9-46ef-9844-2a31cb4df87a admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.052017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6c4da-631a-4c06-b633-fb89564a3210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.052247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3c9d8-f5c9-46ef-9844-2a31cb4df87a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.054101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6c4da-631a-4c06-b633-fb89564a3210 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.054628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3c9d8-f5c9-46ef-9844-2a31cb4df87a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.088510 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 630/1252] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.088655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 623/1253] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.089451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6c4da-631a-4c06-b633-fb89564a3210 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.089917 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3c9d8-f5c9-46ef-9844-2a31cb4df87a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.181547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3c9d8-f5c9-46ef-9844-2a31cb4df87a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.188651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6c4da-631a-4c06-b633-fb89564a3210 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.189023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810ad9-40c6-439e-a7e1-10d54903f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.189695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810ad9-40c6-439e-a7e1-10d54903f2ef admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.190082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810ad9-40c6-439e-a7e1-10d54903f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.192639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810ad9-40c6-439e-a7e1-10d54903f2ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.192639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc810ad9-40c6-439e-a7e1-10d54903f2ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.193898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc810ad9-40c6-439e-a7e1-10d54903f2ef admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.205293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195569d-debc-4dc9-9b62-fd05a3595932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.205293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195569d-debc-4dc9-9b62-fd05a3595932 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.205293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195569d-debc-4dc9-9b62-fd05a3595932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.205293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195569d-debc-4dc9-9b62-fd05a3595932 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.205778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195569d-debc-4dc9-9b62-fd05a3595932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.207743 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195569d-debc-4dc9-9b62-fd05a3595932 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.225475 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:50.226785 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 631/1254] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/projects => generated 391 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.227692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810ad9-40c6-439e-a7e1-10d54903f2ef admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.246274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 624/1255] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.248164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195569d-debc-4dc9-9b62-fd05a3595932 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.293240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:50.312524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810ad9-40c6-439e-a7e1-10d54903f2ef admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.330435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195569d-debc-4dc9-9b62-fd05a3595932 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.337263 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f72fe8-b06f-437f-bd1d-d8c6c501bf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.338133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f72fe8-b06f-437f-bd1d-d8c6c501bf9b admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.339122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f72fe8-b06f-437f-bd1d-d8c6c501bf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.339579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f72fe8-b06f-437f-bd1d-d8c6c501bf9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.340410 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f72fe8-b06f-437f-bd1d-d8c6c501bf9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.342882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f72fe8-b06f-437f-bd1d-d8c6c501bf9b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.383776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 625/1256] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.385019 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f72fe8-b06f-437f-bd1d-d8c6c501bf9b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.402406 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262aa93d-c46f-4ad5-a682-becfd173142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.403261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aa93d-c46f-4ad5-a682-becfd173142a admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.403651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aa93d-c46f-4ad5-a682-becfd173142a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.404589 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aa93d-c46f-4ad5-a682-becfd173142a admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.405586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262aa93d-c46f-4ad5-a682-becfd173142a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.409143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262aa93d-c46f-4ad5-a682-becfd173142a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.456646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 632/1257] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/projects => generated 365 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.458722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262aa93d-c46f-4ad5-a682-becfd173142a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.461749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f72fe8-b06f-437f-bd1d-d8c6c501bf9b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.552225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262aa93d-c46f-4ad5-a682-becfd173142a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.554666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087b11b-e5fe-4051-8da6-c02bcc66ed3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.555363 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087b11b-e5fe-4051-8da6-c02bcc66ed3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.556027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087b11b-e5fe-4051-8da6-c02bcc66ed3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.556565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087b11b-e5fe-4051-8da6-c02bcc66ed3a admin admin] PATH_INFO: `/v3/projects/bc773996e55c425885e129de3fbde25f/users/13a8006701a74384aec23f541686b662/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.559421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb799eda-8403-4ac4-b17d-9dc8a8d7fda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.560433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb799eda-8403-4ac4-b17d-9dc8a8d7fda7 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.560941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb799eda-8403-4ac4-b17d-9dc8a8d7fda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.561530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb799eda-8403-4ac4-b17d-9dc8a8d7fda7 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.562519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb799eda-8403-4ac4-b17d-9dc8a8d7fda7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.563820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING py.warnings [None req-cb799eda-8403-4ac4-b17d-9dc8a8d7fda7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:05:50.563820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: warnings.warn(msg) Oct 28 19:05:50.563820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: Oct 28 19:05:50.565416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb799eda-8403-4ac4-b17d-9dc8a8d7fda7 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.574439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087b11b-e5fe-4051-8da6-c02bcc66ed3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc773996e55c425885e129de3fbde25f, user_id=13a8006701a74384aec23f541686b662, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.579423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087b11b-e5fe-4051-8da6-c02bcc66ed3a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.592886 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 633/1258] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/users => generated 302 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.594076 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb799eda-8403-4ac4-b17d-9dc8a8d7fda7 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.602283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 626/1259] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:50 2022] PUT /identity/v3/projects/bc773996e55c425885e129de3fbde25f/users/13a8006701a74384aec23f541686b662/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:50.606222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb799eda-8403-4ac4-b17d-9dc8a8d7fda7 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.611822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087b11b-e5fe-4051-8da6-c02bcc66ed3a admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.628456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:50.695461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087b11b-e5fe-4051-8da6-c02bcc66ed3a admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.700647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af4d30-92bf-48db-9b51-ea3da4184ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.701478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4d30-92bf-48db-9b51-ea3da4184ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.701827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4d30-92bf-48db-9b51-ea3da4184ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.702177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4d30-92bf-48db-9b51-ea3da4184ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.702891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af4d30-92bf-48db-9b51-ea3da4184ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.705785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af4d30-92bf-48db-9b51-ea3da4184ed7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.706048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6e0f9-11a7-4885-bf21-10158e231335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.706706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6e0f9-11a7-4885-bf21-10158e231335 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.708226 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6e0f9-11a7-4885-bf21-10158e231335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.708226 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6e0f9-11a7-4885-bf21-10158e231335 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.708519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6e0f9-11a7-4885-bf21-10158e231335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.710424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6e0f9-11a7-4885-bf21-10158e231335 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.734323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:50.741590 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 634/1260] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.742742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6e0f9-11a7-4885-bf21-10158e231335 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.745878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 627/1261] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.750923 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af4d30-92bf-48db-9b51-ea3da4184ed7 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.759846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6e0f9-11a7-4885-bf21-10158e231335 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.766190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec509d4-f80a-41db-9e73-a97ade409dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.767542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec509d4-f80a-41db-9e73-a97ade409dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.767967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec509d4-f80a-41db-9e73-a97ade409dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.768245 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec509d4-f80a-41db-9e73-a97ade409dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.768883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec509d4-f80a-41db-9e73-a97ade409dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.770330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af4d30-92bf-48db-9b51-ea3da4184ed7 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.772188 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec509d4-f80a-41db-9e73-a97ade409dc5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.776406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc12742-2764-47a4-8f92-854deaf5129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.777022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc12742-2764-47a4-8f92-854deaf5129f admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.777295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc12742-2764-47a4-8f92-854deaf5129f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.777559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc12742-2764-47a4-8f92-854deaf5129f admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.778188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc12742-2764-47a4-8f92-854deaf5129f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.780155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc12742-2764-47a4-8f92-854deaf5129f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.811143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 635/1262] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/users => generated 284 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.811375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 628/1263] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/users => generated 289 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.811486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc12742-2764-47a4-8f92-854deaf5129f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.812222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec509d4-f80a-41db-9e73-a97ade409dc5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.823673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc12742-2764-47a4-8f92-854deaf5129f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.826274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec509d4-f80a-41db-9e73-a97ade409dc5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.829298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1b6f3-c97f-4587-b3a3-d60f2da94102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.830088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1b6f3-c97f-4587-b3a3-d60f2da94102 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.830369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1b6f3-c97f-4587-b3a3-d60f2da94102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.830675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1b6f3-c97f-4587-b3a3-d60f2da94102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.831116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dfb10-01c4-4b07-8970-cb947d5f9bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.831729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dfb10-01c4-4b07-8970-cb947d5f9bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.831997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1b6f3-c97f-4587-b3a3-d60f2da94102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.832268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dfb10-01c4-4b07-8970-cb947d5f9bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.833169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dfb10-01c4-4b07-8970-cb947d5f9bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.833813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dfb10-01c4-4b07-8970-cb947d5f9bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.834522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1b6f3-c97f-4587-b3a3-d60f2da94102 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.835834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dfb10-01c4-4b07-8970-cb947d5f9bb6 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.846770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 629/1264] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:50.848299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff7925-7d83-40cf-bd80-823aa3c8db3d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.848584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff7925-7d83-40cf-bd80-823aa3c8db3d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.848847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff7925-7d83-40cf-bd80-823aa3c8db3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.871079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 636/1265] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/users => generated 283 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:50.872942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dfb10-01c4-4b07-8970-cb947d5f9bb6 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.885027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-7eff7925-7d83-40cf-bd80-823aa3c8db3d None None] MFA Rules not processed for user `13a8006701a74384aec23f541686b662`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:50.886294 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7eff7925-7d83-40cf-bd80-823aa3c8db3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:50.889724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dfb10-01c4-4b07-8970-cb947d5f9bb6 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.897673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21748745-8f60-447b-8daf-6f80902e8620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.898289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21748745-8f60-447b-8daf-6f80902e8620 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.898563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21748745-8f60-447b-8daf-6f80902e8620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.898831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21748745-8f60-447b-8daf-6f80902e8620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.899503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21748745-8f60-447b-8daf-6f80902e8620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.901092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21748745-8f60-447b-8daf-6f80902e8620 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.914620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 637/1266] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:50.914620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21748745-8f60-447b-8daf-6f80902e8620 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.925056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21748745-8f60-447b-8daf-6f80902e8620 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.931223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3b9db-4262-455f-ac8b-c7a992a880e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.931829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3b9db-4262-455f-ac8b-c7a992a880e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.932204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3b9db-4262-455f-ac8b-c7a992a880e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.932499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3b9db-4262-455f-ac8b-c7a992a880e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.933265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3b9db-4262-455f-ac8b-c7a992a880e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.935906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3b9db-4262-455f-ac8b-c7a992a880e5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.951991 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 638/1267] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:50.956338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3b9db-4262-455f-ac8b-c7a992a880e5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:50.969318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3b9db-4262-455f-ac8b-c7a992a880e5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:50.976062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb24417-3b8e-412f-962b-413c60743330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:50.976868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb24417-3b8e-412f-962b-413c60743330 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:50.977196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb24417-3b8e-412f-962b-413c60743330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:50.977526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb24417-3b8e-412f-962b-413c60743330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:50.978348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb24417-3b8e-412f-962b-413c60743330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:50.980972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb24417-3b8e-412f-962b-413c60743330 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:50.990731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 630/1268] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:50.991879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff7925-7d83-40cf-bd80-823aa3c8db3d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.007644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 639/1269] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.007644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb24417-3b8e-412f-962b-413c60743330 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.009669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff7925-7d83-40cf-bd80-823aa3c8db3d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.020768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511c4d8-6955-4228-afc5-bd97c8e00357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.023380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511c4d8-6955-4228-afc5-bd97c8e00357 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.023665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511c4d8-6955-4228-afc5-bd97c8e00357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.023940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511c4d8-6955-4228-afc5-bd97c8e00357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.026454 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb24417-3b8e-412f-962b-413c60743330 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.029317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511c4d8-6955-4228-afc5-bd97c8e00357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.031185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511c4d8-6955-4228-afc5-bd97c8e00357 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.031643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a218dfb-fba3-4a08-b700-02e6a607dff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.032324 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a218dfb-fba3-4a08-b700-02e6a607dff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.032666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a218dfb-fba3-4a08-b700-02e6a607dff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.032999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a218dfb-fba3-4a08-b700-02e6a607dff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.034288 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a218dfb-fba3-4a08-b700-02e6a607dff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.036392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a218dfb-fba3-4a08-b700-02e6a607dff5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.052440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 640/1270] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.052440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a218dfb-fba3-4a08-b700-02e6a607dff5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.056078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 631/1271] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.056078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511c4d8-6955-4228-afc5-bd97c8e00357 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.071741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a218dfb-fba3-4a08-b700-02e6a607dff5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.077330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46b3ce-075d-497a-b05a-ed3ceace9f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.078451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46b3ce-075d-497a-b05a-ed3ceace9f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.078729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46b3ce-075d-497a-b05a-ed3ceace9f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.079063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46b3ce-075d-497a-b05a-ed3ceace9f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.079713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46b3ce-075d-497a-b05a-ed3ceace9f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.081318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46b3ce-075d-497a-b05a-ed3ceace9f46 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.084073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511c4d8-6955-4228-afc5-bd97c8e00357 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.089644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe4416-553c-468d-a219-3dfb61a2a133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.090251 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe4416-553c-468d-a219-3dfb61a2a133 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.090540 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe4416-553c-468d-a219-3dfb61a2a133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.090807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe4416-553c-468d-a219-3dfb61a2a133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.091484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe4416-553c-468d-a219-3dfb61a2a133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.093104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe4416-553c-468d-a219-3dfb61a2a133 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.094696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 641/1272] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.096248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46b3ce-075d-497a-b05a-ed3ceace9f46 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.109420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46b3ce-075d-497a-b05a-ed3ceace9f46 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.111631 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 632/1273] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.112805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe4416-553c-468d-a219-3dfb61a2a133 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.119530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f34562-d056-4765-b537-cd93cb2aa0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.120257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f34562-d056-4765-b537-cd93cb2aa0cb admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.120629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f34562-d056-4765-b537-cd93cb2aa0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.120991 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f34562-d056-4765-b537-cd93cb2aa0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.121871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f34562-d056-4765-b537-cd93cb2aa0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.123884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f34562-d056-4765-b537-cd93cb2aa0cb admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.127634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe4416-553c-468d-a219-3dfb61a2a133 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.132828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36439575-bc83-493a-a6ab-2ea7886ccbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.133597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36439575-bc83-493a-a6ab-2ea7886ccbf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.134002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36439575-bc83-493a-a6ab-2ea7886ccbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.134366 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36439575-bc83-493a-a6ab-2ea7886ccbf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.135136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36439575-bc83-493a-a6ab-2ea7886ccbf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.136942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 642/1274] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.137313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36439575-bc83-493a-a6ab-2ea7886ccbf5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.141323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f34562-d056-4765-b537-cd93cb2aa0cb admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.153616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f34562-d056-4765-b537-cd93cb2aa0cb admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.163618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42087ecf-1575-443b-b48e-682a6eb9986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.164239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42087ecf-1575-443b-b48e-682a6eb9986e admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.164574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42087ecf-1575-443b-b48e-682a6eb9986e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.164913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42087ecf-1575-443b-b48e-682a6eb9986e admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.166306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42087ecf-1575-443b-b48e-682a6eb9986e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.167953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42087ecf-1575-443b-b48e-682a6eb9986e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.177840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 633/1275] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:51 2022] POST /identity/v3/projects => generated 357 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:51.178943 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36439575-bc83-493a-a6ab-2ea7886ccbf5 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.182148 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 643/1276] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.183011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42087ecf-1575-443b-b48e-682a6eb9986e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.193409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42087ecf-1575-443b-b48e-682a6eb9986e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.211088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:51.299485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44cabd-b8e5-4724-a37b-07ada77946ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.300116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44cabd-b8e5-4724-a37b-07ada77946ec admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.300397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44cabd-b8e5-4724-a37b-07ada77946ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.300668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44cabd-b8e5-4724-a37b-07ada77946ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.301364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44cabd-b8e5-4724-a37b-07ada77946ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.303035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44cabd-b8e5-4724-a37b-07ada77946ec admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.305064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36439575-bc83-493a-a6ab-2ea7886ccbf5 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.315217 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b21ba9-78b0-44fd-b16f-4428939ae22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.316355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b21ba9-78b0-44fd-b16f-4428939ae22f admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.316355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b21ba9-78b0-44fd-b16f-4428939ae22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.317119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b21ba9-78b0-44fd-b16f-4428939ae22f admin admin] PATH_INFO: `/v3/projects/ad951e1cf8214108a93861289153e74d/users/6be71f6545d14cc4ae44404fb9436e6c/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.327285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 644/1277] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.327285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44cabd-b8e5-4724-a37b-07ada77946ec admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.335238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b21ba9-78b0-44fd-b16f-4428939ae22f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad951e1cf8214108a93861289153e74d, user_id=6be71f6545d14cc4ae44404fb9436e6c, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.337895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b21ba9-78b0-44fd-b16f-4428939ae22f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.338818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44cabd-b8e5-4724-a37b-07ada77946ec admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.345072 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba804504-6039-4d9f-9947-ae1445c8e315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.345768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba804504-6039-4d9f-9947-ae1445c8e315 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.346051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba804504-6039-4d9f-9947-ae1445c8e315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.346347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba804504-6039-4d9f-9947-ae1445c8e315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.346972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba804504-6039-4d9f-9947-ae1445c8e315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.348703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba804504-6039-4d9f-9947-ae1445c8e315 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.361516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 634/1278] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:51 2022] PUT /identity/v3/projects/ad951e1cf8214108a93861289153e74d/users/6be71f6545d14cc4ae44404fb9436e6c/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:51.361516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b21ba9-78b0-44fd-b16f-4428939ae22f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.361791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 645/1279] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.361791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba804504-6039-4d9f-9947-ae1445c8e315 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.377912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba804504-6039-4d9f-9947-ae1445c8e315 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.415124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:51.449270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ee827-337a-4227-a68c-fbb54aa8f5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.450258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ee827-337a-4227-a68c-fbb54aa8f5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.450705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ee827-337a-4227-a68c-fbb54aa8f5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.451003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ee827-337a-4227-a68c-fbb54aa8f5b9 admin admin] PATH_INFO: `/v3/projects/4f6c133d9594455182ac1b9ce13df65c/users/c8ca34fc5c5743d0b5e3f4f97ae285e0/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.452716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b21ba9-78b0-44fd-b16f-4428939ae22f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.458665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5516bf3-a987-403b-8531-d79bd259d72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.459372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5516bf3-a987-403b-8531-d79bd259d72d admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.459652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5516bf3-a987-403b-8531-d79bd259d72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.459928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5516bf3-a987-403b-8531-d79bd259d72d admin admin] PATH_INFO: `/v3/projects/526aaf60dd6c42808951c5f48f8fabd2/users/a25c8936b368477880f429e818a321fa/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.471481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ee827-337a-4227-a68c-fbb54aa8f5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f6c133d9594455182ac1b9ce13df65c, user_id=c8ca34fc5c5743d0b5e3f4f97ae285e0, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.474627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ee827-337a-4227-a68c-fbb54aa8f5b9 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.476803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5516bf3-a987-403b-8531-d79bd259d72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=526aaf60dd6c42808951c5f48f8fabd2, user_id=a25c8936b368477880f429e818a321fa, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.479465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5516bf3-a987-403b-8531-d79bd259d72d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.490532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 646/1280] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:51 2022] PUT /identity/v3/projects/4f6c133d9594455182ac1b9ce13df65c/users/c8ca34fc5c5743d0b5e3f4f97ae285e0/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:51.491098 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ee827-337a-4227-a68c-fbb54aa8f5b9 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.495825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 635/1281] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:51 2022] PUT /identity/v3/projects/526aaf60dd6c42808951c5f48f8fabd2/users/a25c8936b368477880f429e818a321fa/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:51.496620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5516bf3-a987-403b-8531-d79bd259d72d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.561421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5516bf3-a987-403b-8531-d79bd259d72d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.561897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ee827-337a-4227-a68c-fbb54aa8f5b9 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.566262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e81f2-51ca-47d1-ad21-f904b95fd911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.567472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e81f2-51ca-47d1-ad21-f904b95fd911 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.567760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e81f2-51ca-47d1-ad21-f904b95fd911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.568030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e81f2-51ca-47d1-ad21-f904b95fd911 admin admin] PATH_INFO: `/v3/projects/811aa5f5fb1f435ba5818bb036af71c9/users/a7f8ea7586344505a95123eeff7327ed/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.568832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458fb73-ce3b-4581-be3f-4f02b65401b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.569600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458fb73-ce3b-4581-be3f-4f02b65401b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.570044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458fb73-ce3b-4581-be3f-4f02b65401b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.570692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458fb73-ce3b-4581-be3f-4f02b65401b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.572396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458fb73-ce3b-4581-be3f-4f02b65401b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.578461 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458fb73-ce3b-4581-be3f-4f02b65401b9 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.588522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e81f2-51ca-47d1-ad21-f904b95fd911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811aa5f5fb1f435ba5818bb036af71c9, user_id=a7f8ea7586344505a95123eeff7327ed, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.591375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e81f2-51ca-47d1-ad21-f904b95fd911 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.608189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 647/1282] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:51 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:51.608352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 636/1283] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:51 2022] PUT /identity/v3/projects/811aa5f5fb1f435ba5818bb036af71c9/users/a7f8ea7586344505a95123eeff7327ed/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:51.608469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458fb73-ce3b-4581-be3f-4f02b65401b9 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.608575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae4a61-5ba7-4253-874e-08dd338f165b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.608962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae4a61-5ba7-4253-874e-08dd338f165b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.608962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae4a61-5ba7-4253-874e-08dd338f165b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.619955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458fb73-ce3b-4581-be3f-4f02b65401b9 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.624128 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:51.631890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-0fae4a61-5ba7-4253-874e-08dd338f165b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:51.635565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0fae4a61-5ba7-4253-874e-08dd338f165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:51.699912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28150e01-0f07-4ada-8bbc-f2fe4615ac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.700976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28150e01-0f07-4ada-8bbc-f2fe4615ac2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.701258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28150e01-0f07-4ada-8bbc-f2fe4615ac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.701531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28150e01-0f07-4ada-8bbc-f2fe4615ac2e admin admin] PATH_INFO: `/v3/projects/67626d51a9484e9c98d0a4758707b896/users/095a332a446b4c6a97257bb7c76f1886/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.723582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28150e01-0f07-4ada-8bbc-f2fe4615ac2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=67626d51a9484e9c98d0a4758707b896, user_id=095a332a446b4c6a97257bb7c76f1886, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.728785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 637/1284] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:51.728903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28150e01-0f07-4ada-8bbc-f2fe4615ac2e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.733318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae4a61-5ba7-4253-874e-08dd338f165b None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.744143 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae4a61-5ba7-4253-874e-08dd338f165b None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.754818 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa68ea-5c2b-46d4-ab36-84f54076ae33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.755570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa68ea-5c2b-46d4-ab36-84f54076ae33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.755904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa68ea-5c2b-46d4-ab36-84f54076ae33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.756380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa68ea-5c2b-46d4-ab36-84f54076ae33 admin admin] PATH_INFO: `/v3/projects/db0080087caa4aa18f189cd2ea6b2032/users/63ea5b25b70849818c41920a167330a4/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.775200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa68ea-5c2b-46d4-ab36-84f54076ae33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0080087caa4aa18f189cd2ea6b2032, user_id=63ea5b25b70849818c41920a167330a4, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.779969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 648/1285] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:51 2022] PUT /identity/v3/projects/67626d51a9484e9c98d0a4758707b896/users/095a332a446b4c6a97257bb7c76f1886/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:51.779969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e36e00-261d-4864-bbd6-7bad9de971f7 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.780229 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa68ea-5c2b-46d4-ab36-84f54076ae33 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.780322 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e36e00-261d-4864-bbd6-7bad9de971f7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.780322 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e36e00-261d-4864-bbd6-7bad9de971f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.797907 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 638/1286] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:51 2022] PUT /identity/v3/projects/db0080087caa4aa18f189cd2ea6b2032/users/63ea5b25b70849818c41920a167330a4/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:51.807077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcfa27-c1fb-4bd7-8d45-b4da8b917ccf None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.807389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcfa27-c1fb-4bd7-8d45-b4da8b917ccf None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.807781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcfa27-c1fb-4bd7-8d45-b4da8b917ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.817502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a3e36e00-261d-4864-bbd6-7bad9de971f7 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:51.819129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a3e36e00-261d-4864-bbd6-7bad9de971f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:51.847190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:51.849717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-38fcfa27-c1fb-4bd7-8d45-b4da8b917ccf None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:51.851438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-38fcfa27-c1fb-4bd7-8d45-b4da8b917ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:51.895529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 649/1287] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:51.896669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e36e00-261d-4864-bbd6-7bad9de971f7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.914099 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e36e00-261d-4864-bbd6-7bad9de971f7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:51.919252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdec5a-7e8d-452a-b075-4c15cd71858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:51.920862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdec5a-7e8d-452a-b075-4c15cd71858d admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.921161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdec5a-7e8d-452a-b075-4c15cd71858d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.921440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdec5a-7e8d-452a-b075-4c15cd71858d admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.922088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdec5a-7e8d-452a-b075-4c15cd71858d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:51.930700 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdec5a-7e8d-452a-b075-4c15cd71858d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:51.934315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 639/1288] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:51.935782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6f2c5-3166-454b-9038-095d606014d1 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:51.936164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6f2c5-3166-454b-9038-095d606014d1 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:51.936539 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6f2c5-3166-454b-9038-095d606014d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:51.961223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 650/1289] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:51.961223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdec5a-7e8d-452a-b075-4c15cd71858d admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:51.985538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-9bd6f2c5-3166-454b-9038-095d606014d1 None None] MFA Rules not processed for user `a7f8ea7586344505a95123eeff7327ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:51.986773 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9bd6f2c5-3166-454b-9038-095d606014d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:52.046379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdec5a-7e8d-452a-b075-4c15cd71858d admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:52.053153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff01ed1-3d0e-4a14-ba7f-d1ccd35cf7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:52.053908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff01ed1-3d0e-4a14-ba7f-d1ccd35cf7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.054291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff01ed1-3d0e-4a14-ba7f-d1ccd35cf7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.055173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff01ed1-3d0e-4a14-ba7f-d1ccd35cf7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.081496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff01ed1-3d0e-4a14-ba7f-d1ccd35cf7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:52.083166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff01ed1-3d0e-4a14-ba7f-d1ccd35cf7b9 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:52.111089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 651/1290] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:05:52.112630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776aa4c5-2968-4d92-b97a-bbba84b6748b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.112924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776aa4c5-2968-4d92-b97a-bbba84b6748b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.113200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776aa4c5-2968-4d92-b97a-bbba84b6748b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.129876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 640/1291] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:52.129876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431beded-41ba-4a60-ba6d-e01d98f3498f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.129876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431beded-41ba-4a60-ba6d-e01d98f3498f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.129876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431beded-41ba-4a60-ba6d-e01d98f3498f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.167013 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-776aa4c5-2968-4d92-b97a-bbba84b6748b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:52.168300 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-776aa4c5-2968-4d92-b97a-bbba84b6748b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:52.172606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:52.177456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-431beded-41ba-4a60-ba6d-e01d98f3498f None None] MFA Rules not processed for user `63ea5b25b70849818c41920a167330a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:52.180342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-431beded-41ba-4a60-ba6d-e01d98f3498f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:52.181885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 652/1292] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:52.182793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776aa4c5-2968-4d92-b97a-bbba84b6748b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:52.200178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776aa4c5-2968-4d92-b97a-bbba84b6748b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:52.212670 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29555f82-73f5-4313-9051-42495bb0d3a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:52.213786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29555f82-73f5-4313-9051-42495bb0d3a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.214378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29555f82-73f5-4313-9051-42495bb0d3a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.214860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29555f82-73f5-4313-9051-42495bb0d3a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.225767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29555f82-73f5-4313-9051-42495bb0d3a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:52.227430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29555f82-73f5-4313-9051-42495bb0d3a0 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:52.240952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 653/1293] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:05:52.241993 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29555f82-73f5-4313-9051-42495bb0d3a0 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:52.265385 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29555f82-73f5-4313-9051-42495bb0d3a0 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:52.269207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc51bb-adb2-4583-8f62-4122a41f52f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:52.270397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc51bb-adb2-4583-8f62-4122a41f52f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.270678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc51bb-adb2-4583-8f62-4122a41f52f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.270945 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc51bb-adb2-4583-8f62-4122a41f52f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.297752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 641/1294] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:52.301867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc51bb-adb2-4583-8f62-4122a41f52f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:52.302020 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431beded-41ba-4a60-ba6d-e01d98f3498f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:52.302120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc51bb-adb2-4583-8f62-4122a41f52f8 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:52.313688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431beded-41ba-4a60-ba6d-e01d98f3498f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:52.321837 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 654/1295] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:05:52.322745 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc51bb-adb2-4583-8f62-4122a41f52f8 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:52.323151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd0e45-1f80-4413-872b-966e017c17ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:52.324866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd0e45-1f80-4413-872b-966e017c17ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.324866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd0e45-1f80-4413-872b-966e017c17ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.325981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd0e45-1f80-4413-872b-966e017c17ef admin admin] PATH_INFO: `/v3/projects/dc75e4459537414699b9e1a31e52f36b/users/b2631c7962bc42e7964dc0afecad1566/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.348378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd0e45-1f80-4413-872b-966e017c17ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc75e4459537414699b9e1a31e52f36b, user_id=b2631c7962bc42e7964dc0afecad1566, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:52.351727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd0e45-1f80-4413-872b-966e017c17ef admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:52.375622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 642/1296] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:52 2022] PUT /identity/v3/projects/dc75e4459537414699b9e1a31e52f36b/users/b2631c7962bc42e7964dc0afecad1566/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:52.377078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd0e45-1f80-4413-872b-966e017c17ef admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:52.472239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc51bb-adb2-4583-8f62-4122a41f52f8 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:52.503590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd0e45-1f80-4413-872b-966e017c17ef admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:52.516711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcee62-6301-41ec-b44d-b7c6679d4720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:52.517566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcee62-6301-41ec-b44d-b7c6679d4720 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.517850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcee62-6301-41ec-b44d-b7c6679d4720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.518123 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcee62-6301-41ec-b44d-b7c6679d4720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.590360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:52.609280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba7578-718e-4c0f-ae2f-dde0d5f09bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:52.609928 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba7578-718e-4c0f-ae2f-dde0d5f09bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.610204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba7578-718e-4c0f-ae2f-dde0d5f09bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.610477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba7578-718e-4c0f-ae2f-dde0d5f09bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.700951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcee62-6301-41ec-b44d-b7c6679d4720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:52.709310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcee62-6301-41ec-b44d-b7c6679d4720 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:52.725345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 643/1297] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:05:52.727153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcee62-6301-41ec-b44d-b7c6679d4720 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:52.738836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcee62-6301-41ec-b44d-b7c6679d4720 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:52.745158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f3741-186b-4725-8f9e-37dba673021f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:52.745779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f3741-186b-4725-8f9e-37dba673021f admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.746863 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f3741-186b-4725-8f9e-37dba673021f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.747179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f3741-186b-4725-8f9e-37dba673021f admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.747930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f3741-186b-4725-8f9e-37dba673021f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:52.750195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f3741-186b-4725-8f9e-37dba673021f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:52.766154 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba7578-718e-4c0f-ae2f-dde0d5f09bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:52.767633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba7578-718e-4c0f-ae2f-dde0d5f09bf8 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:52.792007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 655/1298] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:05:52.792909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba7578-718e-4c0f-ae2f-dde0d5f09bf8 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:52.800073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 644/1299] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:52 2022] POST /identity/v3/projects => generated 363 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:52.801150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f3741-186b-4725-8f9e-37dba673021f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:52.815327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:52.819159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba7578-718e-4c0f-ae2f-dde0d5f09bf8 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:52.949501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1636590-5c94-451a-ae8e-92413ee719ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:52.950410 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1636590-5c94-451a-ae8e-92413ee719ad admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:52.950813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1636590-5c94-451a-ae8e-92413ee719ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:52.951249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1636590-5c94-451a-ae8e-92413ee719ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:52.952239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1636590-5c94-451a-ae8e-92413ee719ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:52.954052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1636590-5c94-451a-ae8e-92413ee719ad admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:52.977818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 656/1300] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:52.980973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1636590-5c94-451a-ae8e-92413ee719ad admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:52.988869 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f3741-186b-4725-8f9e-37dba673021f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:52.998818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1636590-5c94-451a-ae8e-92413ee719ad admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:53.002667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaa16a-bf3f-4351-8465-d5816fa97aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:53.003524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaa16a-bf3f-4351-8465-d5816fa97aca admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.003664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaa16a-bf3f-4351-8465-d5816fa97aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.003872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaa16a-bf3f-4351-8465-d5816fa97aca admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.005059 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eaa16a-bf3f-4351-8465-d5816fa97aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:53.007944 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad741657-ed80-41ff-9fcb-aa287e466cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:53.009070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad741657-ed80-41ff-9fcb-aa287e466cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.009442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad741657-ed80-41ff-9fcb-aa287e466cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.009823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad741657-ed80-41ff-9fcb-aa287e466cc4 admin admin] PATH_INFO: `/v3/projects/dc75e4459537414699b9e1a31e52f36b/users/b2631c7962bc42e7964dc0afecad1566/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.011105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eaa16a-bf3f-4351-8465-d5816fa97aca admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:53.012127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad741657-ed80-41ff-9fcb-aa287e466cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc75e4459537414699b9e1a31e52f36b, user_id=b2631c7962bc42e7964dc0afecad1566, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:53.015439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad741657-ed80-41ff-9fcb-aa287e466cc4 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:53.027997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 657/1301] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:52 2022] PUT /identity/v3/projects/dc75e4459537414699b9e1a31e52f36b/users/b2631c7962bc42e7964dc0afecad1566/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:53.038872 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973a49e-0683-402a-a627-8557ecc2193c None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.039351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973a49e-0683-402a-a627-8557ecc2193c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.039841 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973a49e-0683-402a-a627-8557ecc2193c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.062571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 645/1302] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:52 2022] POST /identity/v3/users => generated 288 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:53.065670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaa16a-bf3f-4351-8465-d5816fa97aca admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:53.088029 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaa16a-bf3f-4351-8465-d5816fa97aca admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:53.106240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8973a49e-0683-402a-a627-8557ecc2193c None None] MFA Rules not processed for user `b2631c7962bc42e7964dc0afecad1566`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:53.110145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8973a49e-0683-402a-a627-8557ecc2193c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:53.218008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 658/1303] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:53 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:53.231779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973a49e-0683-402a-a627-8557ecc2193c None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:53.253703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:53.273072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f7842-91e3-4405-b27b-d0f15cd91694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:53.277732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f7842-91e3-4405-b27b-d0f15cd91694 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.278151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f7842-91e3-4405-b27b-d0f15cd91694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.280134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f7842-91e3-4405-b27b-d0f15cd91694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.282471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f7842-91e3-4405-b27b-d0f15cd91694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:53.291689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f7842-91e3-4405-b27b-d0f15cd91694 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:53.328628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 646/1304] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:53.333289 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f7842-91e3-4405-b27b-d0f15cd91694 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:53.355835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973a49e-0683-402a-a627-8557ecc2193c None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:53.356796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f7842-91e3-4405-b27b-d0f15cd91694 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:53.362530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb3359-b9ed-47f2-94d8-80cf74ac2967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:53.364494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb3359-b9ed-47f2-94d8-80cf74ac2967 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.364897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb3359-b9ed-47f2-94d8-80cf74ac2967 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.365786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb3359-b9ed-47f2-94d8-80cf74ac2967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.369441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f35dda-2dd9-4bd5-958c-e754020d9616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:53.370210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f35dda-2dd9-4bd5-958c-e754020d9616 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.371154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f35dda-2dd9-4bd5-958c-e754020d9616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.371963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f35dda-2dd9-4bd5-958c-e754020d9616 admin admin] PATH_INFO: `/v3/projects/b87af532ca1a4150b48201e8770b2c99/users/cb3ab0cf101845bea7b1f5081c1e2709/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.408080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb3359-b9ed-47f2-94d8-80cf74ac2967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:53.412446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb3359-b9ed-47f2-94d8-80cf74ac2967 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:53.412604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f35dda-2dd9-4bd5-958c-e754020d9616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87af532ca1a4150b48201e8770b2c99, user_id=cb3ab0cf101845bea7b1f5081c1e2709, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:53.418228 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f35dda-2dd9-4bd5-958c-e754020d9616 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:53.444345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 659/1305] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:53 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:05:53.451029 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 647/1306] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:53 2022] PUT /identity/v3/projects/b87af532ca1a4150b48201e8770b2c99/users/cb3ab0cf101845bea7b1f5081c1e2709/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:05:53.452914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb3359-b9ed-47f2-94d8-80cf74ac2967 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:53.479822 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb3359-b9ed-47f2-94d8-80cf74ac2967 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:53.595293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3cf31-3d5f-48fd-a687-ace8e8afd69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:53.595869 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3cf31-3d5f-48fd-a687-ace8e8afd69e admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.595951 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3cf31-3d5f-48fd-a687-ace8e8afd69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.596232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3cf31-3d5f-48fd-a687-ace8e8afd69e admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.596897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3cf31-3d5f-48fd-a687-ace8e8afd69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:53.600648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3cf31-3d5f-48fd-a687-ace8e8afd69e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:53.601119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f35dda-2dd9-4bd5-958c-e754020d9616 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:53.614047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 660/1307] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:53.620243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3cf31-3d5f-48fd-a687-ace8e8afd69e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:53.636062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3cf31-3d5f-48fd-a687-ace8e8afd69e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:53.642908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f5cb6-a590-4e8b-bfa1-c86f499441c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:53.644240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f5cb6-a590-4e8b-bfa1-c86f499441c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.644931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f5cb6-a590-4e8b-bfa1-c86f499441c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.645356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f5cb6-a590-4e8b-bfa1-c86f499441c1 admin admin] PATH_INFO: `/v3/projects/b87af532ca1a4150b48201e8770b2c99/users/cb3ab0cf101845bea7b1f5081c1e2709/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.649832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f5cb6-a590-4e8b-bfa1-c86f499441c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87af532ca1a4150b48201e8770b2c99, user_id=cb3ab0cf101845bea7b1f5081c1e2709, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:53.653551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f5cb6-a590-4e8b-bfa1-c86f499441c1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:53.670503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 661/1308] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:53 2022] PUT /identity/v3/projects/b87af532ca1a4150b48201e8770b2c99/users/cb3ab0cf101845bea7b1f5081c1e2709/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:53.681310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0b749-86e4-4005-a9f8-a34a51ee7599 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.681748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0b749-86e4-4005-a9f8-a34a51ee7599 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.682281 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0b749-86e4-4005-a9f8-a34a51ee7599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.730355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-9dc0b749-86e4-4005-a9f8-a34a51ee7599 None None] MFA Rules not processed for user `cb3ab0cf101845bea7b1f5081c1e2709`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:53.731659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f35dda-2dd9-4bd5-958c-e754020d9616 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:53.732071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9dc0b749-86e4-4005-a9f8-a34a51ee7599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:53.772569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:53.834721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908554bd-aed3-4293-9adc-6a64f1674c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:53.835409 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908554bd-aed3-4293-9adc-6a64f1674c5f service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:53.867293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908554bd-aed3-4293-9adc-6a64f1674c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:53.867636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908554bd-aed3-4293-9adc-6a64f1674c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:53.918438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 662/1309] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:05:54.028149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0b749-86e4-4005-a9f8-a34a51ee7599 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:54.034386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908554bd-aed3-4293-9adc-6a64f1674c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:54.036386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908554bd-aed3-4293-9adc-6a64f1674c5f service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:54.042747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0b749-86e4-4005-a9f8-a34a51ee7599 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:54.060611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf9dfc-a0df-447a-84ea-6f0341a5aa6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:54.065802 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf9dfc-a0df-447a-84ea-6f0341a5aa6d service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:54.065802 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf9dfc-a0df-447a-84ea-6f0341a5aa6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:54.066030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf9dfc-a0df-447a-84ea-6f0341a5aa6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:54.068748 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 648/1310] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:05:54.101885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf9dfc-a0df-447a-84ea-6f0341a5aa6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:54.103572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf9dfc-a0df-447a-84ea-6f0341a5aa6d service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:54.123481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 663/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:05:54.301778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908554bd-aed3-4293-9adc-6a64f1674c5f service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:54.313315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908554bd-aed3-4293-9adc-6a64f1674c5f service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:54.333532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b05029-4bf8-4fe3-a24c-cd9330333634 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:54.334162 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b05029-4bf8-4fe3-a24c-cd9330333634 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:54.334670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b05029-4bf8-4fe3-a24c-cd9330333634 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:54.334670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b05029-4bf8-4fe3-a24c-cd9330333634 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:54.495370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b05029-4bf8-4fe3-a24c-cd9330333634 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:54.497001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b05029-4bf8-4fe3-a24c-cd9330333634 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:54.516601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 649/1312] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:05:55.353030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ddd80-b065-4487-97a7-5690e7789d4a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:55.353947 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ddd80-b065-4487-97a7-5690e7789d4a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:55.354436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ddd80-b065-4487-97a7-5690e7789d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:55.365291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:55.413447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-c54ddd80-b065-4487-97a7-5690e7789d4a None None] MFA Rules not processed for user `6be71f6545d14cc4ae44404fb9436e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:55.414692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c54ddd80-b065-4487-97a7-5690e7789d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:55.511688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd74587-d7bb-47d3-a3fb-a764c9a8d9f1 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:55.512306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd74587-d7bb-47d3-a3fb-a764c9a8d9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:55.512306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd74587-d7bb-47d3-a3fb-a764c9a8d9f1 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:55.513844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 650/1313] 10.222.0.58 () {54 vars in 961 bytes} [Fri Oct 28 19:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 19:05:55.534955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ffcc7-cd7c-455d-82dc-cfdcabceed4b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:55.535275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ffcc7-cd7c-455d-82dc-cfdcabceed4b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:55.535683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ffcc7-cd7c-455d-82dc-cfdcabceed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:55.567392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 664/1314] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:55 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:55.568455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ddd80-b065-4487-97a7-5690e7789d4a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:55.694588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-058ffcc7-cd7c-455d-82dc-cfdcabceed4b None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:55.700944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-058ffcc7-cd7c-455d-82dc-cfdcabceed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:55.774078 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ddd80-b065-4487-97a7-5690e7789d4a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:55.811320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7deca4f-0fb1-44e0-ad1f-136ca7f88abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:55.813291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7deca4f-0fb1-44e0-ad1f-136ca7f88abc admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:55.815246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7deca4f-0fb1-44e0-ad1f-136ca7f88abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:55.816284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7deca4f-0fb1-44e0-ad1f-136ca7f88abc admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:55.821088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7deca4f-0fb1-44e0-ad1f-136ca7f88abc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:55.823225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7deca4f-0fb1-44e0-ad1f-136ca7f88abc admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:55.876113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:55.895359 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 651/1315] 10.222.0.58 () {58 vars in 1058 bytes} [Fri Oct 28 19:05:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:55.906240 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b5666-eccc-41c5-a6df-c1442a93b42c None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:55.909338 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b5666-eccc-41c5-a6df-c1442a93b42c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:55.909620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b5666-eccc-41c5-a6df-c1442a93b42c None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:55.913319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 652/1316] 10.222.0.58 () {54 vars in 912 bytes} [Fri Oct 28 19:05:55 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:05:55.915557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 665/1317] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:05:55 2022] POST /identity/v3/projects => generated 405 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:55.918145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b5666-eccc-41c5-a6df-c1442a93b42c None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:55.922006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cb831-e184-48c7-8219-1996efff18cc None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:55.922403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cb831-e184-48c7-8219-1996efff18cc None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:55.924144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cb831-e184-48c7-8219-1996efff18cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:55.990757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-157cb831-e184-48c7-8219-1996efff18cc None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:55.997566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-157cb831-e184-48c7-8219-1996efff18cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:56.056285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b5666-eccc-41c5-a6df-c1442a93b42c None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:56.063369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779d074-48f0-497a-a349-fc00fe0a79c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:56.064667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779d074-48f0-497a-a349-fc00fe0a79c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:56.065523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779d074-48f0-497a-a349-fc00fe0a79c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:56.066110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779d074-48f0-497a-a349-fc00fe0a79c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:56.066948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779d074-48f0-497a-a349-fc00fe0a79c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:56.070013 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779d074-48f0-497a-a349-fc00fe0a79c4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:56.142568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 653/1318] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:05:55 2022] POST /identity/v3/users => generated 309 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:05:56.145896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779d074-48f0-497a-a349-fc00fe0a79c4 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:56.178291 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779d074-48f0-497a-a349-fc00fe0a79c4 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:56.203668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd752b-76b0-4ac9-8027-0c0b05bac550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:56.204744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd752b-76b0-4ac9-8027-0c0b05bac550 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:56.205554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd752b-76b0-4ac9-8027-0c0b05bac550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:56.205988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd752b-76b0-4ac9-8027-0c0b05bac550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:56.207507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd752b-76b0-4ac9-8027-0c0b05bac550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:56.209358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd752b-76b0-4ac9-8027-0c0b05bac550 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:56.213177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 666/1319] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:05:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:56.230196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cb831-e184-48c7-8219-1996efff18cc None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:56.232315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 654/1320] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:05:56.242591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd752b-76b0-4ac9-8027-0c0b05bac550 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:56.264934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:56.274500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd752b-76b0-4ac9-8027-0c0b05bac550 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:56.286924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381f17a-a3e8-4e6f-8c28-1a3f71479093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:56.288742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381f17a-a3e8-4e6f-8c28-1a3f71479093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:56.289256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381f17a-a3e8-4e6f-8c28-1a3f71479093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:56.291490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381f17a-a3e8-4e6f-8c28-1a3f71479093 admin admin] PATH_INFO: `/v3/projects/c7da4ed91f4643a5ab6dc3cae03f4a2d/users/875ab6e0fe09451fb6e175e7283820f2/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:56.334423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381f17a-a3e8-4e6f-8c28-1a3f71479093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7da4ed91f4643a5ab6dc3cae03f4a2d, user_id=875ab6e0fe09451fb6e175e7283820f2, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:56.343362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381f17a-a3e8-4e6f-8c28-1a3f71479093 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:56.345474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cb831-e184-48c7-8219-1996efff18cc None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:56.352007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb49fb-7d4b-4739-8a42-267efe4dfc89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:56.353283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb49fb-7d4b-4739-8a42-267efe4dfc89 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:56.353848 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb49fb-7d4b-4739-8a42-267efe4dfc89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:56.354126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb49fb-7d4b-4739-8a42-267efe4dfc89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:56.375829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 655/1321] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:56 2022] PUT /identity/v3/projects/c7da4ed91f4643a5ab6dc3cae03f4a2d/users/875ab6e0fe09451fb6e175e7283820f2/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:05:56.380889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381f17a-a3e8-4e6f-8c28-1a3f71479093 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:56.383511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb49fb-7d4b-4739-8a42-267efe4dfc89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:56.385212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb49fb-7d4b-4739-8a42-267efe4dfc89 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:56.404324 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 667/1322] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:05:56.564133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381f17a-a3e8-4e6f-8c28-1a3f71479093 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:56.582161 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4a9ff-de69-4152-bb6b-e40fedb826bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:56.583374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4a9ff-de69-4152-bb6b-e40fedb826bb admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:56.586171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4a9ff-de69-4152-bb6b-e40fedb826bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:56.586711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4a9ff-de69-4152-bb6b-e40fedb826bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:56.587756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4a9ff-de69-4152-bb6b-e40fedb826bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:56.590265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4a9ff-de69-4152-bb6b-e40fedb826bb admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:56.630405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 656/1323] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:05:56.634514 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb49fb-7d4b-4739-8a42-267efe4dfc89 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:56.649271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb49fb-7d4b-4739-8a42-267efe4dfc89 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:56.658445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca710cd7-5fb4-4ad6-884c-ef74f15f0380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:56.659581 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca710cd7-5fb4-4ad6-884c-ef74f15f0380 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:56.663713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca710cd7-5fb4-4ad6-884c-ef74f15f0380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:56.664254 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca710cd7-5fb4-4ad6-884c-ef74f15f0380 admin admin] PATH_INFO: `/v3/projects/c7da4ed91f4643a5ab6dc3cae03f4a2d/users/875ab6e0fe09451fb6e175e7283820f2/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:56.666579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca710cd7-5fb4-4ad6-884c-ef74f15f0380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7da4ed91f4643a5ab6dc3cae03f4a2d, user_id=875ab6e0fe09451fb6e175e7283820f2, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:56.669566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca710cd7-5fb4-4ad6-884c-ef74f15f0380 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:56.705525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 668/1324] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:05:56 2022] PUT /identity/v3/projects/c7da4ed91f4643a5ab6dc3cae03f4a2d/users/875ab6e0fe09451fb6e175e7283820f2/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:05:56.732589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012a8ed-0670-42d0-9e7a-50f926ee7256 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:56.733224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012a8ed-0670-42d0-9e7a-50f926ee7256 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:56.733688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012a8ed-0670-42d0-9e7a-50f926ee7256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:56.812224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-7012a8ed-0670-42d0-9e7a-50f926ee7256 None None] MFA Rules not processed for user `e868bb74b33f467380562aec01a3d3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:56.828104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7012a8ed-0670-42d0-9e7a-50f926ee7256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:56.902608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:05:56.966088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 657/1325] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:05:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:05:56.996534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a7728-b819-447f-a236-0354cb1c8b67 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:56.997218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a7728-b819-447f-a236-0354cb1c8b67 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:56.997698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a7728-b819-447f-a236-0354cb1c8b67 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:57.007986 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 669/1326] 10.222.0.58 () {54 vars in 961 bytes} [Fri Oct 28 19:05:56 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:05:57.016781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55454d-e276-4bbb-bb00-5b3b6651ad57 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:57.017419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55454d-e276-4bbb-bb00-5b3b6651ad57 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:57.017897 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55454d-e276-4bbb-bb00-5b3b6651ad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:57.075208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-fa55454d-e276-4bbb-bb00-5b3b6651ad57 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:05:57.078672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-fa55454d-e276-4bbb-bb00-5b3b6651ad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:05:57.278146 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 658/1327] 10.222.0.58 () {58 vars in 1058 bytes} [Fri Oct 28 19:05:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:05:57.295177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a7728-b819-447f-a236-0354cb1c8b67 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:05:57.324559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a7728-b819-447f-a236-0354cb1c8b67 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:05:57.343559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ab718-fa91-4f86-9566-b2f4b0d0f61d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:05:57.344547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ab718-fa91-4f86-9566-b2f4b0d0f61d service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:05:57.345010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ab718-fa91-4f86-9566-b2f4b0d0f61d service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:05:57.350001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ab718-fa91-4f86-9566-b2f4b0d0f61d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:05:57.366813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ab718-fa91-4f86-9566-b2f4b0d0f61d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:05:57.372288 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ab718-fa91-4f86-9566-b2f4b0d0f61d service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:05:57.425659 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 670/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:05:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:00.300655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b90af3-37e2-49c5-9073-1487b0406320 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:00.301677 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b90af3-37e2-49c5-9073-1487b0406320 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:00.302160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b90af3-37e2-49c5-9073-1487b0406320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:00.366956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b5b90af3-37e2-49c5-9073-1487b0406320 None None] MFA Rules not processed for user `875ab6e0fe09451fb6e175e7283820f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:00.369333 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b5b90af3-37e2-49c5-9073-1487b0406320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:00.547354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 659/1329] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:00 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:00.783529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2301e11-4e12-4a2c-ad94-258c32ca0150 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:00.784725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2301e11-4e12-4a2c-ad94-258c32ca0150 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:00.785175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2301e11-4e12-4a2c-ad94-258c32ca0150 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:00.786709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 671/1330] 10.222.0.58 () {54 vars in 912 bytes} [Fri Oct 28 19:06:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:06:00.793284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439eb60-8902-4cb7-aec5-0f65c236e197 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:00.793801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439eb60-8902-4cb7-aec5-0f65c236e197 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:00.794214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439eb60-8902-4cb7-aec5-0f65c236e197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:00.833352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a439eb60-8902-4cb7-aec5-0f65c236e197 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:00.835492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a439eb60-8902-4cb7-aec5-0f65c236e197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:00.862046 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 660/1331] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:06:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:00.879868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2301e11-4e12-4a2c-ad94-258c32ca0150 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:01.011038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2301e11-4e12-4a2c-ad94-258c32ca0150 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:01.015306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e318f9-1b78-496b-98d6-cf1e0f85c60e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:01.015934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e318f9-1b78-496b-98d6-cf1e0f85c60e service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:01.016213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e318f9-1b78-496b-98d6-cf1e0f85c60e service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:01.016548 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e318f9-1b78-496b-98d6-cf1e0f85c60e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:01.043298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e318f9-1b78-496b-98d6-cf1e0f85c60e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:01.044748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e318f9-1b78-496b-98d6-cf1e0f85c60e service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:01.064979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 672/1332] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:06:02.903111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73da991-7810-40fa-9087-9377dc5139da None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:02.903575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73da991-7810-40fa-9087-9377dc5139da None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:02.905851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73da991-7810-40fa-9087-9377dc5139da None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:02.907772 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 661/1333] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:06:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:06:02.948070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00e25a-b604-45ae-8a98-d815462d6a06 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:02.948692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00e25a-b604-45ae-8a98-d815462d6a06 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:02.948692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00e25a-b604-45ae-8a98-d815462d6a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:03.051092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-9b00e25a-b604-45ae-8a98-d815462d6a06 None None] MFA Rules not processed for user `da958657ac3041848d24a76b76705489`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:03.052751 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9b00e25a-b604-45ae-8a98-d815462d6a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:03.252219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 673/1334] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:06:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:03.303747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73da991-7810-40fa-9087-9377dc5139da None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:03.349888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73da991-7810-40fa-9087-9377dc5139da None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:03.361128 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8b947-f75d-47c0-8b8b-72c214b6b2b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da958657ac3041848d24a76b76705489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:03.364168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8b947-f75d-47c0-8b8b-72c214b6b2b7 service placement] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:03.365644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8b947-f75d-47c0-8b8b-72c214b6b2b7 service placement] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:03.366950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8b947-f75d-47c0-8b8b-72c214b6b2b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:03.393664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8b947-f75d-47c0-8b8b-72c214b6b2b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:03.396179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8b947-f75d-47c0-8b8b-72c214b6b2b7 service placement] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:03.426542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 662/1335] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:03.546856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ad13a-43fd-41d3-8fc6-51621b38c36a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:03.547183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ad13a-43fd-41d3-8fc6-51621b38c36a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:03.547625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ad13a-43fd-41d3-8fc6-51621b38c36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:03.583476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-978ad13a-43fd-41d3-8fc6-51621b38c36a None None] MFA Rules not processed for user `e868bb74b33f467380562aec01a3d3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:03.588661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-978ad13a-43fd-41d3-8fc6-51621b38c36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:03.607320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 674/1336] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:05.200063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8b947-f75d-47c0-8b8b-72c214b6b2b7 service placement] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:05.263476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:05.302761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8b947-f75d-47c0-8b8b-72c214b6b2b7 service placement] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:05.308365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:05.310205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:05.310876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:05.311415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] PATH_INFO: `/v3/users/6be71f6545d14cc4ae44404fb9436e6c` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:05.320150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6be71f6545d14cc4ae44404fb9436e6c)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:05.324579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: WARNING py.warnings [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:06:05.324579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: warnings.warn(msg) Oct 28 19:06:05.324579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: Oct 28 19:06:05.326771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:05.535969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6be71f6545d14cc4ae44404fb9436e6c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:05.538757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6be71f6545d14cc4ae44404fb9436e6c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:05.594244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6be71f6545d14cc4ae44404fb9436e6c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:05.630348 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6be71f6545d14cc4ae44404fb9436e6c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:05.657347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6c279580-1b69-4c55-9e21-bc19ea28eccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6be71f6545d14cc4ae44404fb9436e6c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:05.705576 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 663/1337] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:05 2022] DELETE /identity/v3/users/6be71f6545d14cc4ae44404fb9436e6c => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:06.209954 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ad13a-43fd-41d3-8fc6-51621b38c36a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:06.239174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ad13a-43fd-41d3-8fc6-51621b38c36a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:06.263636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebbba5-7cd4-4576-8116-0f3033982073 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:06.264264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebbba5-7cd4-4576-8116-0f3033982073 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:06.264264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebbba5-7cd4-4576-8116-0f3033982073 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:06.266062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 664/1338] 10.222.0.58 () {54 vars in 912 bytes} [Fri Oct 28 19:06:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:06:06.278675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae2f29-8e92-4077-9b64-4ffc55916a8b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:06.279386 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae2f29-8e92-4077-9b64-4ffc55916a8b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:06.279849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae2f29-8e92-4077-9b64-4ffc55916a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:06.289614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:06.356758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-8eae2f29-8e92-4077-9b64-4ffc55916a8b None None] MFA Rules not processed for user `9cffb662f4104e349c81c578bf66ab7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:06.358116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8eae2f29-8e92-4077-9b64-4ffc55916a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:06.423100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:06.424177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:06.424727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:06.425264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] PATH_INFO: `/v3/projects/ad951e1cf8214108a93861289153e74d` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:06.426914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad951e1cf8214108a93861289153e74d)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:06.428697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: WARNING py.warnings [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 19:06:06.428697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: warnings.warn(msg) Oct 28 19:06:06.428697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: Oct 28 19:06:06.430623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:06.467639 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 665/1339] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:06:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:06.523350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad951e1cf8214108a93861289153e74d', 'request_id': 'req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.540016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad951e1cf8214108a93861289153e74d', 'request_id': 'req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.560965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad951e1cf8214108a93861289153e74d', 'request_id': 'req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.589731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] The token cache is being invalidate because project ad951e1cf8214108a93861289153e74d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:06.590579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.591392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.598294 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae2f29-8e92-4077-9b64-4ffc55916a8b None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:06.600626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8eae2f29-8e92-4077-9b64-4ffc55916a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:06.609251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 675/1340] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:06 2022] DELETE /identity/v3/projects/ad951e1cf8214108a93861289153e74d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:06.617743 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae2f29-8e92-4077-9b64-4ffc55916a8b None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:06.698902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:06.727075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:06.742212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:06.780612 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:06.784001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:06.787110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:06.787394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] PATH_INFO: `/v3/users/875ab6e0fe09451fb6e175e7283820f2` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:06.791998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=875ab6e0fe09451fb6e175e7283820f2)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:06.804674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:06.924752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875ab6e0fe09451fb6e175e7283820f2'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.926129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875ab6e0fe09451fb6e175e7283820f2'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.947193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6281a3-d6f4-488a-b853-436ed8c2b5f6 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:06.952172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:06.952920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:06.956871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:06.956871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:06.958094 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875ab6e0fe09451fb6e175e7283820f2'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.958437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:06.972944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875ab6e0fe09451fb6e175e7283820f2'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.978784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:06.980978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:06.990729 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875ab6e0fe09451fb6e175e7283820f2'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:06.994778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 676/1341] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:06.995990 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:06.998055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:06.999579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 666/1342] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:06 2022] DELETE /identity/v3/users/875ab6e0fe09451fb6e175e7283820f2 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:07.048963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f71e7-188c-4b20-a519-c76b1c01b18a service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:07.126028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:07.138273 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:07.139048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:07.139458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:07.139914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] PATH_INFO: `/v3/users/a7f8ea7586344505a95123eeff7327ed` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:07.141843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f8ea7586344505a95123eeff7327ed)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:07.148429 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:07.257351 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f8ea7586344505a95123eeff7327ed'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.258061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f8ea7586344505a95123eeff7327ed'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.289451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f8ea7586344505a95123eeff7327ed'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.307480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f8ea7586344505a95123eeff7327ed'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.310266 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5c51363d-d521-4795-a30d-ea65e7c1c2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f8ea7586344505a95123eeff7327ed'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.312526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:07.314021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:07.341564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 677/1343] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:06 2022] DELETE /identity/v3/users/a7f8ea7586344505a95123eeff7327ed => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:07.346950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799dceb-5d68-4bc8-a412-fd65dd861219 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:07.347938 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799dceb-5d68-4bc8-a412-fd65dd861219 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:07.348137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799dceb-5d68-4bc8-a412-fd65dd861219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:07.353781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b72a79-71da-473a-a1b7-17deda8047ec admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:07.371482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-7799dceb-5d68-4bc8-a412-fd65dd861219 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:07.372535 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7799dceb-5d68-4bc8-a412-fd65dd861219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:07.540574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 678/1344] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:07.542840 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665d6d9-ad2b-43f6-a27f-67d15a20320e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:07.543159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665d6d9-ad2b-43f6-a27f-67d15a20320e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:07.543435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665d6d9-ad2b-43f6-a27f-67d15a20320e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:07.545267 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:07.548083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:07.548369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:07.548643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] PATH_INFO: `/v3/projects/c7da4ed91f4643a5ab6dc3cae03f4a2d` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:07.551055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7da4ed91f4643a5ab6dc3cae03f4a2d)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:07.553696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:07.617417 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6665d6d9-ad2b-43f6-a27f-67d15a20320e None None] MFA Rules not processed for user `095a332a446b4c6a97257bb7c76f1886`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:07.622585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6665d6d9-ad2b-43f6-a27f-67d15a20320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:07.680525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7da4ed91f4643a5ab6dc3cae03f4a2d', 'request_id': 'req-aa32b11a-092d-4091-b424-4f09edbc9659'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.697619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7da4ed91f4643a5ab6dc3cae03f4a2d', 'request_id': 'req-aa32b11a-092d-4091-b424-4f09edbc9659'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.711193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7da4ed91f4643a5ab6dc3cae03f4a2d', 'request_id': 'req-aa32b11a-092d-4091-b424-4f09edbc9659'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.743420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:07.763945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] The token cache is being invalidate because project c7da4ed91f4643a5ab6dc3cae03f4a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:07.764342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.764881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:07.772777 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 667/1345] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:07 2022] DELETE /identity/v3/projects/c7da4ed91f4643a5ab6dc3cae03f4a2d => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:07.773723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:07.775435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:07.802784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 679/1346] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:07.802784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7715aa-0766-4407-9aa9-beb4f4f4fa40 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:07.802784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7715aa-0766-4407-9aa9-beb4f4f4fa40 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:07.802784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7715aa-0766-4407-9aa9-beb4f4f4fa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:07.816818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:07.840106 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ce7715aa-0766-4407-9aa9-beb4f4f4fa40 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:07.841307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ce7715aa-0766-4407-9aa9-beb4f4f4fa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:07.918478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32b11a-092d-4091-b424-4f09edbc9659 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:07.931383 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7646a7-4c0b-411c-b349-cd6d2d46d914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:07.932629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7646a7-4c0b-411c-b349-cd6d2d46d914 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:07.932915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7646a7-4c0b-411c-b349-cd6d2d46d914 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:07.933206 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7646a7-4c0b-411c-b349-cd6d2d46d914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:07.935520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5d7646a7-4c0b-411c-b349-cd6d2d46d914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:07.975686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 680/1347] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:07.981068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a12ec-f4ca-4aca-87c4-85f096ae3a75 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:07.983020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a12ec-f4ca-4aca-87c4-85f096ae3a75 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:07.983310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a12ec-f4ca-4aca-87c4-85f096ae3a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:08.064332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-5f3a12ec-f4ca-4aca-87c4-85f096ae3a75 None None] MFA Rules not processed for user `095a332a446b4c6a97257bb7c76f1886`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:08.065496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5f3a12ec-f4ca-4aca-87c4-85f096ae3a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:08.141663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7646a7-4c0b-411c-b349-cd6d2d46d914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:08.143754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7646a7-4c0b-411c-b349-cd6d2d46d914 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:08.161543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 668/1348] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:08.164960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b4662-9a3f-4bb7-affa-88562aaac192 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:08.165489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b4662-9a3f-4bb7-affa-88562aaac192 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:08.165587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b4662-9a3f-4bb7-affa-88562aaac192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:08.194187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2b1b4662-9a3f-4bb7-affa-88562aaac192 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:08.213795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2b1b4662-9a3f-4bb7-affa-88562aaac192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:08.223583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 681/1349] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:08.238356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a12ec-f4ca-4aca-87c4-85f096ae3a75 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:08.239963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5f3a12ec-f4ca-4aca-87c4-85f096ae3a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:08.246671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 669/1350] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:08.249293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b4662-9a3f-4bb7-affa-88562aaac192 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:08.303358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b4662-9a3f-4bb7-affa-88562aaac192 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:08.319285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2986181-162a-45e4-b8e4-8e70b4845ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:08.321411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2986181-162a-45e4-b8e4-8e70b4845ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:08.322275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2986181-162a-45e4-b8e4-8e70b4845ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:08.322822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2986181-162a-45e4-b8e4-8e70b4845ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:08.327078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2986181-162a-45e4-b8e4-8e70b4845ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:08.327759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2986181-162a-45e4-b8e4-8e70b4845ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:08.332321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2986181-162a-45e4-b8e4-8e70b4845ac6 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:08.376485 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 670/1351] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:08.377579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2986181-162a-45e4-b8e4-8e70b4845ac6 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:08.403610 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2986181-162a-45e4-b8e4-8e70b4845ac6 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:08.415870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599bf22-afc0-4d4f-ba29-38cde6cd7de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:08.417135 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599bf22-afc0-4d4f-ba29-38cde6cd7de9 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:08.417422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599bf22-afc0-4d4f-ba29-38cde6cd7de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:08.417709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599bf22-afc0-4d4f-ba29-38cde6cd7de9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:08.418673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599bf22-afc0-4d4f-ba29-38cde6cd7de9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:08.425451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599bf22-afc0-4d4f-ba29-38cde6cd7de9 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:08.468497 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a12ec-f4ca-4aca-87c4-85f096ae3a75 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:08.487536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:08.488366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:08.488672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:08.489115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:08.492262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:08.521873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:08.523782 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:08.525213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 671/1352] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:08 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:08.529725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599bf22-afc0-4d4f-ba29-38cde6cd7de9 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:08.567308 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 682/1353] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:06:08.813272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599bf22-afc0-4d4f-ba29-38cde6cd7de9 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:08.830318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:08.832384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:08.841919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adf6f9-2749-4f9d-9718-2e072a10d955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:08.846262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adf6f9-2749-4f9d-9718-2e072a10d955 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:08.846603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adf6f9-2749-4f9d-9718-2e072a10d955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:08.846878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adf6f9-2749-4f9d-9718-2e072a10d955 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:08.847617 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adf6f9-2749-4f9d-9718-2e072a10d955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:08.856087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adf6f9-2749-4f9d-9718-2e072a10d955 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:08.894268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24c978-f8fe-4133-bb44-b440f00f92a0 service nova] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:08.912946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:08.913602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:08.913887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:08.914149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] PATH_INFO: `/v3/users/63ea5b25b70849818c41920a167330a4` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:08.915438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ea5b25b70849818c41920a167330a4)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:08.917387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:08.929262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:08.951141 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 672/1354] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:08 2022] POST /identity/v3/users => generated 285 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:08.952071 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adf6f9-2749-4f9d-9718-2e072a10d955 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:08.958178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e7adf6f9-2749-4f9d-9718-2e072a10d955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:08.986126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adf6f9-2749-4f9d-9718-2e072a10d955 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:08.991092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:08.994321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:08.995214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:08.995357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:08.995516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] PATH_INFO: `/v3/projects/811aa5f5fb1f435ba5818bb036af71c9` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:08.996926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=811aa5f5fb1f435ba5818bb036af71c9)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:08.998758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:09.022025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ea5b25b70849818c41920a167330a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.023146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ea5b25b70849818c41920a167330a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.049487 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ea5b25b70849818c41920a167330a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.064944 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ea5b25b70849818c41920a167330a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.065765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ea5b25b70849818c41920a167330a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.077889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811aa5f5fb1f435ba5818bb036af71c9', 'request_id': 'req-143631ad-fe40-4d01-aa0d-6924184fb030'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.087438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811aa5f5fb1f435ba5818bb036af71c9', 'request_id': 'req-143631ad-fe40-4d01-aa0d-6924184fb030'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.091072 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 683/1355] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:08 2022] DELETE /identity/v3/users/63ea5b25b70849818c41920a167330a4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:09.137886 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:09.137886 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:09.142425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811aa5f5fb1f435ba5818bb036af71c9', 'request_id': 'req-143631ad-fe40-4d01-aa0d-6924184fb030'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.142425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] The token cache is being invalidate because project 811aa5f5fb1f435ba5818bb036af71c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:09.142425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.142425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:09.150401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 673/1356] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:08 2022] DELETE /identity/v3/projects/811aa5f5fb1f435ba5818bb036af71c9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:09.150401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:09.152285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:09.250922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e0bcd-aa75-43ff-949f-c1e2a68afb1a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:09.270757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be540dd6-d7ac-4069-a67b-99c3603e25bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da958657ac3041848d24a76b76705489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:09.271759 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be540dd6-d7ac-4069-a67b-99c3603e25bf service placement] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.272045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be540dd6-d7ac-4069-a67b-99c3603e25bf service placement] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.272323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be540dd6-d7ac-4069-a67b-99c3603e25bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.277370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-be540dd6-d7ac-4069-a67b-99c3603e25bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:09.292955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be540dd6-d7ac-4069-a67b-99c3603e25bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:09.294690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be540dd6-d7ac-4069-a67b-99c3603e25bf service placement] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:09.335404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 684/1357] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:09.336948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9f651-0474-4f92-9ee7-c9734a4f840b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.337368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9f651-0474-4f92-9ee7-c9734a4f840b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.337773 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9f651-0474-4f92-9ee7-c9734a4f840b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.359106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143631ad-fe40-4d01-aa0d-6924184fb030 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:09.363149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-97b9f651-0474-4f92-9ee7-c9734a4f840b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:09.364645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-97b9f651-0474-4f92-9ee7-c9734a4f840b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:09.547902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 685/1358] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:09.550416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10939d84-69c9-4332-b126-077c016936eb None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.559741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10939d84-69c9-4332-b126-077c016936eb None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.561966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10939d84-69c9-4332-b126-077c016936eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.597777 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f0f41-1128-4e5f-8ebb-9dbde9861381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:09.599851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f0f41-1128-4e5f-8ebb-9dbde9861381 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.600260 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f0f41-1128-4e5f-8ebb-9dbde9861381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.600798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f0f41-1128-4e5f-8ebb-9dbde9861381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.601684 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f0f41-1128-4e5f-8ebb-9dbde9861381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:09.603558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f0f41-1128-4e5f-8ebb-9dbde9861381 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:09.626518 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-10939d84-69c9-4332-b126-077c016936eb None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:09.630010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-10939d84-69c9-4332-b126-077c016936eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:09.640721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 674/1359] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:09.641703 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f0f41-1128-4e5f-8ebb-9dbde9861381 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:09.647784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5e9f0f41-1128-4e5f-8ebb-9dbde9861381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:09.653465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 686/1360] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:09.664556 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4481-613d-4cf4-9183-e40dc9d38697 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.664840 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4481-613d-4cf4-9183-e40dc9d38697 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.665109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4481-613d-4cf4-9183-e40dc9d38697 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.672193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 687/1361] 10.222.0.58 () {54 vars in 912 bytes} [Fri Oct 28 19:06:09 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:06:09.674552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af4481-613d-4cf4-9183-e40dc9d38697 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:09.703016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:09.725305 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af4481-613d-4cf4-9183-e40dc9d38697 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:09.745660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebba095-3593-4e9e-a086-ad53ba888386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:09.751111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebba095-3593-4e9e-a086-ad53ba888386 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.751445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebba095-3593-4e9e-a086-ad53ba888386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.751722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebba095-3593-4e9e-a086-ad53ba888386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.752396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebba095-3593-4e9e-a086-ad53ba888386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:09.759546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebba095-3593-4e9e-a086-ad53ba888386 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:09.771755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f0f41-1128-4e5f-8ebb-9dbde9861381 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:09.783026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 688/1362] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:09.783736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ced70c-c628-4152-84c7-0fcc001bd0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:09.784402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ced70c-c628-4152-84c7-0fcc001bd0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.784778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ced70c-c628-4152-84c7-0fcc001bd0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.785060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ced70c-c628-4152-84c7-0fcc001bd0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.795200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba4eef-c1d6-4c5a-9cbc-9ecdafc74749 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.795651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba4eef-c1d6-4c5a-9cbc-9ecdafc74749 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.795941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba4eef-c1d6-4c5a-9cbc-9ecdafc74749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.808283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ced70c-c628-4152-84c7-0fcc001bd0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:09.811768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ced70c-c628-4152-84c7-0fcc001bd0c6 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:09.825739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 675/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:09.827173 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f876efd-f873-4221-bfdd-ccf182458620 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.827463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f876efd-f873-4221-bfdd-ccf182458620 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.828202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f876efd-f873-4221-bfdd-ccf182458620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.859994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-1f876efd-f873-4221-bfdd-ccf182458620 None None] MFA Rules not processed for user `9cffb662f4104e349c81c578bf66ab7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:09.861569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1f876efd-f873-4221-bfdd-ccf182458620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:09.870895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 676/1364] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:06:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:09.871976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f876efd-f873-4221-bfdd-ccf182458620 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:09.878882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f7ba4eef-c1d6-4c5a-9cbc-9ecdafc74749 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:09.883321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f7ba4eef-c1d6-4c5a-9cbc-9ecdafc74749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:09.885849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f876efd-f873-4221-bfdd-ccf182458620 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:09.894379 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dafd05-d571-4b10-bbc2-672d1af7d37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:09.897823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dafd05-d571-4b10-bbc2-672d1af7d37f admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.898185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dafd05-d571-4b10-bbc2-672d1af7d37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.898512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dafd05-d571-4b10-bbc2-672d1af7d37f admin admin] PATH_INFO: `/v3/projects/1fc247cec6b24c849e15ccf1fa480e88/users/42b30bc1e6c14efcb24a7cdbd40be3af/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.921154 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 689/1365] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:09.922216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba4eef-c1d6-4c5a-9cbc-9ecdafc74749 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:09.923541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dafd05-d571-4b10-bbc2-672d1af7d37f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc247cec6b24c849e15ccf1fa480e88, user_id=42b30bc1e6c14efcb24a7cdbd40be3af, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:09.930997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dafd05-d571-4b10-bbc2-672d1af7d37f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:09.939281 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba4eef-c1d6-4c5a-9cbc-9ecdafc74749 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:09.949092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d37a7d-5387-43f2-92ba-8db3f1026af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:09.973254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 677/1366] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:09 2022] PUT /identity/v3/projects/1fc247cec6b24c849e15ccf1fa480e88/users/42b30bc1e6c14efcb24a7cdbd40be3af/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:09.975332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dafd05-d571-4b10-bbc2-672d1af7d37f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:09.983426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-05dafd05-d571-4b10-bbc2-672d1af7d37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:09.985502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d37a7d-5387-43f2-92ba-8db3f1026af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:09.985502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d37a7d-5387-43f2-92ba-8db3f1026af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:09.986569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d37a7d-5387-43f2-92ba-8db3f1026af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:09.988509 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d37a7d-5387-43f2-92ba-8db3f1026af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:09.990016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d37a7d-5387-43f2-92ba-8db3f1026af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:09.997021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d37a7d-5387-43f2-92ba-8db3f1026af9 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:10.021631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:10.044661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 690/1367] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:10.049632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f420b-0cd4-4002-95a9-6ac93648aafe None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:10.051566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f420b-0cd4-4002-95a9-6ac93648aafe None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:10.051987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f420b-0cd4-4002-95a9-6ac93648aafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:10.140548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dafd05-d571-4b10-bbc2-672d1af7d37f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:10.149324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d4b72-cde0-4b64-b891-91a57cb98d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:10.149846 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d4b72-cde0-4b64-b891-91a57cb98d67 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:10.151957 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d4b72-cde0-4b64-b891-91a57cb98d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:10.152290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d4b72-cde0-4b64-b891-91a57cb98d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:10.185644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d4b72-cde0-4b64-b891-91a57cb98d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:10.189427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2f0f420b-0cd4-4002-95a9-6ac93648aafe None None] MFA Rules not processed for user `42b30bc1e6c14efcb24a7cdbd40be3af`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:10.192358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d4b72-cde0-4b64-b891-91a57cb98d67 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:10.194007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2f0f420b-0cd4-4002-95a9-6ac93648aafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:10.232382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 678/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:10.233802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d4b72-cde0-4b64-b891-91a57cb98d67 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:10.365852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 691/1369] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:10.371971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f420b-0cd4-4002-95a9-6ac93648aafe None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:10.385913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d4b72-cde0-4b64-b891-91a57cb98d67 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:10.392601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd462090-286f-4450-9156-bbc6335f2dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:10.394932 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd462090-286f-4450-9156-bbc6335f2dad admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:10.395294 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd462090-286f-4450-9156-bbc6335f2dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:10.395837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd462090-286f-4450-9156-bbc6335f2dad admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:10.396992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f420b-0cd4-4002-95a9-6ac93648aafe None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:10.397789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd462090-286f-4450-9156-bbc6335f2dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:10.399808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd462090-286f-4450-9156-bbc6335f2dad admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:10.409271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae4ec6-a9b2-4c46-8b25-3cc13366936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:10.410289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae4ec6-a9b2-4c46-8b25-3cc13366936d admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:10.410639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae4ec6-a9b2-4c46-8b25-3cc13366936d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:10.411083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae4ec6-a9b2-4c46-8b25-3cc13366936d admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:10.411864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae4ec6-a9b2-4c46-8b25-3cc13366936d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:10.417394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae4ec6-a9b2-4c46-8b25-3cc13366936d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:10.456028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 692/1370] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:10 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:10.460108 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae4ec6-a9b2-4c46-8b25-3cc13366936d admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:10.476822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 679/1371] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:10 2022] POST /identity/v3/projects => generated 373 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:10.482623 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd462090-286f-4450-9156-bbc6335f2dad admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:10.547910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:10.561676 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae4ec6-a9b2-4c46-8b25-3cc13366936d admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:10.648317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ec078-f047-45aa-974d-626722397d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:10.649389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ec078-f047-45aa-974d-626722397d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:10.649389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ec078-f047-45aa-974d-626722397d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:10.652035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ec078-f047-45aa-974d-626722397d5f admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:10.657808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ec078-f047-45aa-974d-626722397d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:10.658624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ec078-f047-45aa-974d-626722397d5f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:10.670575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd462090-286f-4450-9156-bbc6335f2dad admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:10.690604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe7692-b1ea-4cd4-8653-3f50a8f39240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:10.692323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe7692-b1ea-4cd4-8653-3f50a8f39240 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:10.692926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe7692-b1ea-4cd4-8653-3f50a8f39240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:10.694781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe7692-b1ea-4cd4-8653-3f50a8f39240 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:10.696678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe7692-b1ea-4cd4-8653-3f50a8f39240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:10.706224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe7692-b1ea-4cd4-8653-3f50a8f39240 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:10.708002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 693/1372] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:10 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:10.709048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ec078-f047-45aa-974d-626722397d5f admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:10.714103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d40ec078-f047-45aa-974d-626722397d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:10.735199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:10.742423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ec078-f047-45aa-974d-626722397d5f admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:10.757244 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:10.758055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:10.758430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:10.758796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] PATH_INFO: `/v3/projects/db0080087caa4aa18f189cd2ea6b2032` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:10.760124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db0080087caa4aa18f189cd2ea6b2032)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:10.764524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:10.766343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 680/1373] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:10 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:10.767500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe7692-b1ea-4cd4-8653-3f50a8f39240 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:10.782177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe7692-b1ea-4cd4-8653-3f50a8f39240 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:10.791199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8bdbd-b5f8-4b9a-9c1b-53e4c8de84b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:10.792237 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8bdbd-b5f8-4b9a-9c1b-53e4c8de84b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:10.792529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8bdbd-b5f8-4b9a-9c1b-53e4c8de84b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:10.792913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8bdbd-b5f8-4b9a-9c1b-53e4c8de84b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:10.793652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8bdbd-b5f8-4b9a-9c1b-53e4c8de84b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:10.795659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8bdbd-b5f8-4b9a-9c1b-53e4c8de84b7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:10.816951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 681/1374] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:10.836923 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8bdbd-b5f8-4b9a-9c1b-53e4c8de84b7 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:10.858511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8bdbd-b5f8-4b9a-9c1b-53e4c8de84b7 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:10.892479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db0080087caa4aa18f189cd2ea6b2032', 'request_id': 'req-44167e53-4032-4e7b-b4c3-61ae959ba0f5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:10.926477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db0080087caa4aa18f189cd2ea6b2032', 'request_id': 'req-44167e53-4032-4e7b-b4c3-61ae959ba0f5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:10.942395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db0080087caa4aa18f189cd2ea6b2032', 'request_id': 'req-44167e53-4032-4e7b-b4c3-61ae959ba0f5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:10.980234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] The token cache is being invalidate because project db0080087caa4aa18f189cd2ea6b2032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:10.982393 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:10.984388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:10.988547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 694/1375] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:10 2022] DELETE /identity/v3/projects/db0080087caa4aa18f189cd2ea6b2032 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:10.990408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:10.993092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:10.999734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8b16a-780c-48b7-bff1-0a80eb5781df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:11.000579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8b16a-780c-48b7-bff1-0a80eb5781df admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:11.002555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8b16a-780c-48b7-bff1-0a80eb5781df admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:11.003806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8b16a-780c-48b7-bff1-0a80eb5781df admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:11.004634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8b16a-780c-48b7-bff1-0a80eb5781df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:11.006575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8b16a-780c-48b7-bff1-0a80eb5781df admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:11.026435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44167e53-4032-4e7b-b4c3-61ae959ba0f5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:11.027464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 682/1376] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:11.028853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8b16a-780c-48b7-bff1-0a80eb5781df admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:11.031312 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-99a8b16a-780c-48b7-bff1-0a80eb5781df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:11.032544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66b126-5480-4f24-bfde-d995cf7d97c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:11.033623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66b126-5480-4f24-bfde-d995cf7d97c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:11.033999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66b126-5480-4f24-bfde-d995cf7d97c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:11.036158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66b126-5480-4f24-bfde-d995cf7d97c2 admin admin] PATH_INFO: `/v3/projects/42c71d69ccff44e4aa9ac54741f61caf/users/c7bf3084392d4d37ae57747d65e2d287/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:11.056344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8b16a-780c-48b7-bff1-0a80eb5781df admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:11.067119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:11.068018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:11.068896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:11.068896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] PATH_INFO: `/v3/users/cb3ab0cf101845bea7b1f5081c1e2709` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:11.070573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3ab0cf101845bea7b1f5081c1e2709)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:11.072895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66b126-5480-4f24-bfde-d995cf7d97c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42c71d69ccff44e4aa9ac54741f61caf, user_id=c7bf3084392d4d37ae57747d65e2d287, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:11.073172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:11.076234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66b126-5480-4f24-bfde-d995cf7d97c2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:11.106438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 695/1377] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:10 2022] PUT /identity/v3/projects/42c71d69ccff44e4aa9ac54741f61caf/users/c7bf3084392d4d37ae57747d65e2d287/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:11.110066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66b126-5480-4f24-bfde-d995cf7d97c2 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:11.113138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cb66b126-5480-4f24-bfde-d995cf7d97c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:11.147994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3ab0cf101845bea7b1f5081c1e2709'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:11.148874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3ab0cf101845bea7b1f5081c1e2709'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:11.172692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3ab0cf101845bea7b1f5081c1e2709'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:11.186473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3ab0cf101845bea7b1f5081c1e2709'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:11.205840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3ab0cf101845bea7b1f5081c1e2709'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:11.214821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 683/1378] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:11 2022] DELETE /identity/v3/users/cb3ab0cf101845bea7b1f5081c1e2709 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:11.216032 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:11.220631 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:11.244033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:11.304919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66b126-5480-4f24-bfde-d995cf7d97c2 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:11.450046 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2eb03-7e5f-4d85-a4c3-e5b88ff794b4 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:11.458093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ba4b2-ec5d-4808-a905-81e0697a75e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:11.459180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ba4b2-ec5d-4808-a905-81e0697a75e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:11.459467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ba4b2-ec5d-4808-a905-81e0697a75e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:11.459878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ba4b2-ec5d-4808-a905-81e0697a75e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:11.461407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ba4b2-ec5d-4808-a905-81e0697a75e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:11.464257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ba4b2-ec5d-4808-a905-81e0697a75e5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:11.490203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 684/1379] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:11.494413 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ba4b2-ec5d-4808-a905-81e0697a75e5 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:11.528571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ba4b2-ec5d-4808-a905-81e0697a75e5 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:11.549473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a74e3-71df-40d6-91c1-efb6193611ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:11.551430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a74e3-71df-40d6-91c1-efb6193611ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:11.551853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a74e3-71df-40d6-91c1-efb6193611ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:11.552272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a74e3-71df-40d6-91c1-efb6193611ee admin admin] PATH_INFO: `/v3/projects/42c71d69ccff44e4aa9ac54741f61caf/users/c7bf3084392d4d37ae57747d65e2d287/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:11.557158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a74e3-71df-40d6-91c1-efb6193611ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=42c71d69ccff44e4aa9ac54741f61caf, user_id=c7bf3084392d4d37ae57747d65e2d287, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:11.557740 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cb872-dec3-46cb-b865-52ce15e8d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:11.558533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cb872-dec3-46cb-b865-52ce15e8d037 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:11.558930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cb872-dec3-46cb-b865-52ce15e8d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:11.559778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cb872-dec3-46cb-b865-52ce15e8d037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:11.560542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6cb872-dec3-46cb-b865-52ce15e8d037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:11.561201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a74e3-71df-40d6-91c1-efb6193611ee admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:11.563932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6cb872-dec3-46cb-b865-52ce15e8d037 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:11.595442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 696/1381] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:11.597680 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2195aa-2355-4ec2-8de2-89a5492d8bf9 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:11.599093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2195aa-2355-4ec2-8de2-89a5492d8bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:11.599267 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 685/1381] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:11 2022] PUT /identity/v3/projects/42c71d69ccff44e4aa9ac54741f61caf/users/c7bf3084392d4d37ae57747d65e2d287/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:11.599662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2195aa-2355-4ec2-8de2-89a5492d8bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:11.600904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a74e3-71df-40d6-91c1-efb6193611ee admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:11.602892 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-121a74e3-71df-40d6-91c1-efb6193611ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:11.631401 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:11.712868 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a74e3-71df-40d6-91c1-efb6193611ee admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:11.714040 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8d2195aa-2355-4ec2-8de2-89a5492d8bf9 None None] MFA Rules not processed for user `c7bf3084392d4d37ae57747d65e2d287`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:11.719848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05b520-5cfd-4c27-9b64-a04225fbfcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:11.724349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b520-5cfd-4c27-9b64-a04225fbfcce admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:11.724784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b520-5cfd-4c27-9b64-a04225fbfcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:11.725219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b520-5cfd-4c27-9b64-a04225fbfcce admin admin] PATH_INFO: `/v3/projects/7f15987bc1e84bc78bfbb63e323f00f9/users/f9826eca2619482c9efd54bd803bb122/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:11.725343 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8d2195aa-2355-4ec2-8de2-89a5492d8bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:11.778551 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05b520-5cfd-4c27-9b64-a04225fbfcce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f15987bc1e84bc78bfbb63e323f00f9, user_id=f9826eca2619482c9efd54bd803bb122, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:11.781341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05b520-5cfd-4c27-9b64-a04225fbfcce admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:11.816803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 686/1382] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:11 2022] PUT /identity/v3/projects/7f15987bc1e84bc78bfbb63e323f00f9/users/f9826eca2619482c9efd54bd803bb122/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:11.826619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f8268-73c4-41ae-bde3-2478b63518ff None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:11.827097 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f8268-73c4-41ae-bde3-2478b63518ff None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:11.827655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f8268-73c4-41ae-bde3-2478b63518ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:11.913081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-1f7f8268-73c4-41ae-bde3-2478b63518ff None None] MFA Rules not processed for user `f9826eca2619482c9efd54bd803bb122`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:11.914325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1f7f8268-73c4-41ae-bde3-2478b63518ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:11.965796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 697/1383] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:11.977120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2195aa-2355-4ec2-8de2-89a5492d8bf9 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:11.984059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8d2195aa-2355-4ec2-8de2-89a5492d8bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:12.049732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 687/1384] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:12.105496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2195aa-2355-4ec2-8de2-89a5492d8bf9 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:12.158180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f8268-73c4-41ae-bde3-2478b63518ff None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:12.160250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1f7f8268-73c4-41ae-bde3-2478b63518ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:12.179473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:12.218009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f16aa2-c0ef-4aa6-84f8-8d5765d3b66f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:12.218667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f16aa2-c0ef-4aa6-84f8-8d5765d3b66f service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:12.218943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f16aa2-c0ef-4aa6-84f8-8d5765d3b66f service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:12.219242 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f16aa2-c0ef-4aa6-84f8-8d5765d3b66f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:12.291026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f8268-73c4-41ae-bde3-2478b63518ff None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:12.308754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:12.309542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:12.310617 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:12.311111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] PATH_INFO: `/v3/projects/b87af532ca1a4150b48201e8770b2c99` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:12.312457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b87af532ca1a4150b48201e8770b2c99)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:12.316180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:12.363559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:12.366552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f16aa2-c0ef-4aa6-84f8-8d5765d3b66f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:12.368414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f16aa2-c0ef-4aa6-84f8-8d5765d3b66f service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:12.402868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 698/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:12.418376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f16aa2-c0ef-4aa6-84f8-8d5765d3b66f service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:12.422196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-82f16aa2-c0ef-4aa6-84f8-8d5765d3b66f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:12.482622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b87af532ca1a4150b48201e8770b2c99', 'request_id': 'req-6de0ca6b-c2f9-46cc-826d-07a115566ed4'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:12.505004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b87af532ca1a4150b48201e8770b2c99', 'request_id': 'req-6de0ca6b-c2f9-46cc-826d-07a115566ed4'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:12.524287 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b87af532ca1a4150b48201e8770b2c99', 'request_id': 'req-6de0ca6b-c2f9-46cc-826d-07a115566ed4'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:12.542610 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] The token cache is being invalidate because project b87af532ca1a4150b48201e8770b2c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:12.544673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:12.545616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:12.549450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 688/1386] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:12 2022] DELETE /identity/v3/projects/b87af532ca1a4150b48201e8770b2c99 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:12.553643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:12.558398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:12.672083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f16aa2-c0ef-4aa6-84f8-8d5765d3b66f service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:12.741706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0ca6b-c2f9-46cc-826d-07a115566ed4 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:12.759145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c470ba9-a541-40e6-9de6-ed3daba07a8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:12.782525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c470ba9-a541-40e6-9de6-ed3daba07a8c service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:12.783197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c470ba9-a541-40e6-9de6-ed3daba07a8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:12.784163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c470ba9-a541-40e6-9de6-ed3daba07a8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:12.790752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1c470ba9-a541-40e6-9de6-ed3daba07a8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:12.841193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2096c-ef9f-482f-8011-defdfd16f326 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:12.842023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2096c-ef9f-482f-8011-defdfd16f326 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:12.842447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2096c-ef9f-482f-8011-defdfd16f326 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:12.842834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2096c-ef9f-482f-8011-defdfd16f326 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:12.847652 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-acb2096c-ef9f-482f-8011-defdfd16f326 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:12.922766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:12.954248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c470ba9-a541-40e6-9de6-ed3daba07a8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:12.955804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c470ba9-a541-40e6-9de6-ed3daba07a8c service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:12.975064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 689/1387] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:12.976490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4b36-8594-48ad-8c9a-3609d542677c None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:12.976771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4b36-8594-48ad-8c9a-3609d542677c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:12.977617 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4b36-8594-48ad-8c9a-3609d542677c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:13.045714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-08ef4b36-8594-48ad-8c9a-3609d542677c None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:13.046951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-08ef4b36-8594-48ad-8c9a-3609d542677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:13.150382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2096c-ef9f-482f-8011-defdfd16f326 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:13.156251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2096c-ef9f-482f-8011-defdfd16f326 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:13.190050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 690/1388] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:13.191016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95910f-3c20-4241-a64d-3f7122f19498 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:13.191395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95910f-3c20-4241-a64d-3f7122f19498 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:13.191761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95910f-3c20-4241-a64d-3f7122f19498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:13.201547 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 699/1389] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:13.204968 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22ac8f-dec0-4a54-8669-e6f2bfe9b348 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:13.205263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22ac8f-dec0-4a54-8669-e6f2bfe9b348 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:13.206728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22ac8f-dec0-4a54-8669-e6f2bfe9b348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:13.265294 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4b95910f-3c20-4241-a64d-3f7122f19498 None None] MFA Rules not processed for user `a25c8936b368477880f429e818a321fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:13.267582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4b95910f-3c20-4241-a64d-3f7122f19498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:13.299294 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-af22ac8f-dec0-4a54-8669-e6f2bfe9b348 None None] MFA Rules not processed for user `c8ca34fc5c5743d0b5e3f4f97ae285e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:13.307314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-af22ac8f-dec0-4a54-8669-e6f2bfe9b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:13.363262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:13.471834 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 691/1390] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:13.480475 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111d5c6-5881-4d1b-947c-c0d4b256e65d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:13.480475 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111d5c6-5881-4d1b-947c-c0d4b256e65d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:13.480833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111d5c6-5881-4d1b-947c-c0d4b256e65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:13.559202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-0111d5c6-5881-4d1b-947c-c0d4b256e65d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:13.560445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0111d5c6-5881-4d1b-947c-c0d4b256e65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:13.618230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 692/1391] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:13.618230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55c46f-564a-41dd-8e69-c8b401068604 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:13.618230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55c46f-564a-41dd-8e69-c8b401068604 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:13.618230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55c46f-564a-41dd-8e69-c8b401068604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:13.825165 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-cc55c46f-564a-41dd-8e69-c8b401068604 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:13.828279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-cc55c46f-564a-41dd-8e69-c8b401068604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:13.921133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:13.936813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 700/1392] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 735 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:13.938555 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22ac8f-dec0-4a54-8669-e6f2bfe9b348 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:13.941562 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-af22ac8f-dec0-4a54-8669-e6f2bfe9b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:13.974270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22ac8f-dec0-4a54-8669-e6f2bfe9b348 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:13.975173 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 693/1393] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:06:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:13.976047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55c46f-564a-41dd-8e69-c8b401068604 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:13.988164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a089c5d-d293-4c26-b28b-7538c0b0635f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:13.988866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a089c5d-d293-4c26-b28b-7538c0b0635f service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:13.989121 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a089c5d-d293-4c26-b28b-7538c0b0635f service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:13.989303 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a089c5d-d293-4c26-b28b-7538c0b0635f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:13.996075 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55c46f-564a-41dd-8e69-c8b401068604 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:14.004508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a089c5d-d293-4c26-b28b-7538c0b0635f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:14.007671 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a089c5d-d293-4c26-b28b-7538c0b0635f service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:14.011528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d3704-3e24-4309-a5ee-f654dcde2cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:14.012149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d3704-3e24-4309-a5ee-f654dcde2cce admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:14.012689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d3704-3e24-4309-a5ee-f654dcde2cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:14.013055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d3704-3e24-4309-a5ee-f654dcde2cce admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:14.013707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d3704-3e24-4309-a5ee-f654dcde2cce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:14.014008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d3704-3e24-4309-a5ee-f654dcde2cce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:14.015636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d3704-3e24-4309-a5ee-f654dcde2cce admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:14.036259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 701/1394] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:14.037806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693f529-89b5-4d08-a873-ba750d3191bc None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:14.038044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693f529-89b5-4d08-a873-ba750d3191bc None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:14.038158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693f529-89b5-4d08-a873-ba750d3191bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:14.050587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 694/1395] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:14.051793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d3704-3e24-4309-a5ee-f654dcde2cce admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:14.071912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d3704-3e24-4309-a5ee-f654dcde2cce admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:14.078371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a693f529-89b5-4d08-a873-ba750d3191bc None None] MFA Rules not processed for user `c8ca34fc5c5743d0b5e3f4f97ae285e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:14.080100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a693f529-89b5-4d08-a873-ba750d3191bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:14.084871 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfb9a3-b957-46da-81b3-5ca192d6f63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:14.085505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfb9a3-b957-46da-81b3-5ca192d6f63a admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:14.085782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfb9a3-b957-46da-81b3-5ca192d6f63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:14.086052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfb9a3-b957-46da-81b3-5ca192d6f63a admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:14.086701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfb9a3-b957-46da-81b3-5ca192d6f63a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:14.088664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfb9a3-b957-46da-81b3-5ca192d6f63a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:14.101058 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 702/1396] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:14.176420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 695/1397] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:14 2022] POST /identity/v3/projects => generated 399 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:14.177571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693f529-89b5-4d08-a873-ba750d3191bc None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:14.196178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693f529-89b5-4d08-a873-ba750d3191bc None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:14.284795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfb9a3-b957-46da-81b3-5ca192d6f63a admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:14.288036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-33dfb9a3-b957-46da-81b3-5ca192d6f63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:14.371191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8514c-60bd-42d0-aeff-849d2542b081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:14.372143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8514c-60bd-42d0-aeff-849d2542b081 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:14.372615 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8514c-60bd-42d0-aeff-849d2542b081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:14.373134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8514c-60bd-42d0-aeff-849d2542b081 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:14.374097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8514c-60bd-42d0-aeff-849d2542b081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:14.376582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8514c-60bd-42d0-aeff-849d2542b081 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:14.465895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 703/1398] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:14 2022] POST /identity/v3/users => generated 300 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:06:14.467476 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8514c-60bd-42d0-aeff-849d2542b081 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:14.536119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfb9a3-b957-46da-81b3-5ca192d6f63a admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:14.564650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54683b94-1c2e-461c-a1bc-2f97d198e323 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:14.566346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54683b94-1c2e-461c-a1bc-2f97d198e323 service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:14.566577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54683b94-1c2e-461c-a1bc-2f97d198e323 service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:14.566852 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54683b94-1c2e-461c-a1bc-2f97d198e323 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:14.601924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8514c-60bd-42d0-aeff-849d2542b081 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:14.610285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a5a63-166e-41d4-95c0-fa7a45207057 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:14.611087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a5a63-166e-41d4-95c0-fa7a45207057 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:14.611370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a5a63-166e-41d4-95c0-fa7a45207057 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:14.611691 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a5a63-166e-41d4-95c0-fa7a45207057 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:14.651122 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:14.658089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54683b94-1c2e-461c-a1bc-2f97d198e323 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:14.660539 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54683b94-1c2e-461c-a1bc-2f97d198e323 service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:14.672935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 696/1399] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:06:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 28 19:06:14.674457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54683b94-1c2e-461c-a1bc-2f97d198e323 service swift] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:14.687475 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54683b94-1c2e-461c-a1bc-2f97d198e323 service swift] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:14.694678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dff919-2cf0-4ce0-8f80-56b059170794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:14.695514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dff919-2cf0-4ce0-8f80-56b059170794 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:14.695799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dff919-2cf0-4ce0-8f80-56b059170794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:14.696498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dff919-2cf0-4ce0-8f80-56b059170794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:14.698168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dff919-2cf0-4ce0-8f80-56b059170794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:14.699894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dff919-2cf0-4ce0-8f80-56b059170794 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:14.718732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 697/1400] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:14.719839 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dff919-2cf0-4ce0-8f80-56b059170794 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:14.722296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-53dff919-2cf0-4ce0-8f80-56b059170794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:14.744623 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dff919-2cf0-4ce0-8f80-56b059170794 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:14.755692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:14.756746 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:14.757055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:14.757411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] PATH_INFO: `/v3/users/f9826eca2619482c9efd54bd803bb122` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:14.759891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9826eca2619482c9efd54bd803bb122)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:14.765461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:14.767946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a5a63-166e-41d4-95c0-fa7a45207057 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:14.771396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a5a63-166e-41d4-95c0-fa7a45207057 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:14.795751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:14.829070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 704/1401] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:14.830039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a5a63-166e-41d4-95c0-fa7a45207057 service nova] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:14.901455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a5a63-166e-41d4-95c0-fa7a45207057 service nova] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:14.907082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9826eca2619482c9efd54bd803bb122'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:14.912388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9826eca2619482c9efd54bd803bb122'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:14.932441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986b778-fc4d-450e-b42e-95d2cbfece88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:14.933489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986b778-fc4d-450e-b42e-95d2cbfece88 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:14.935855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986b778-fc4d-450e-b42e-95d2cbfece88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:14.936541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986b778-fc4d-450e-b42e-95d2cbfece88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:14.937246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986b778-fc4d-450e-b42e-95d2cbfece88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:14.954125 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986b778-fc4d-450e-b42e-95d2cbfece88 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:14.963388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9826eca2619482c9efd54bd803bb122'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:14.985910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9826eca2619482c9efd54bd803bb122'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:14.989023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 705/1402] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:15.005941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986b778-fc4d-450e-b42e-95d2cbfece88 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:15.010006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b776e226-0fa1-4833-9d83-514ba5e3f618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9826eca2619482c9efd54bd803bb122'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:15.016736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0986b778-fc4d-450e-b42e-95d2cbfece88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:15.024028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 698/1403] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:14 2022] DELETE /identity/v3/users/f9826eca2619482c9efd54bd803bb122 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:15.028595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de42ff6-5f4c-423c-b7e8-f8b6767c998f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:15.028883 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de42ff6-5f4c-423c-b7e8-f8b6767c998f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:15.029265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de42ff6-5f4c-423c-b7e8-f8b6767c998f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:15.067347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-9de42ff6-5f4c-423c-b7e8-f8b6767c998f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:15.069054 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9de42ff6-5f4c-423c-b7e8-f8b6767c998f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:15.077339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986b778-fc4d-450e-b42e-95d2cbfece88 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:15.331928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 699/1404] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:15.372254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de42ff6-5f4c-423c-b7e8-f8b6767c998f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:15.374056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9de42ff6-5f4c-423c-b7e8-f8b6767c998f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:15.408109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ec792-bb5d-4362-8f56-2042c171dbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:15.411656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ec792-bb5d-4362-8f56-2042c171dbe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:15.412214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ec792-bb5d-4362-8f56-2042c171dbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:15.412899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ec792-bb5d-4362-8f56-2042c171dbe4 admin admin] PATH_INFO: `/v3/projects/0a7fa08049a74d6bbb02f67573575d2f/users/9de538cf1e954ca399379c830a4d44d3/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:15.457082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ec792-bb5d-4362-8f56-2042c171dbe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7fa08049a74d6bbb02f67573575d2f, user_id=9de538cf1e954ca399379c830a4d44d3, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:15.460786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ec792-bb5d-4362-8f56-2042c171dbe4 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:15.505061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 706/1405] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:15 2022] PUT /identity/v3/projects/0a7fa08049a74d6bbb02f67573575d2f/users/9de538cf1e954ca399379c830a4d44d3/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:15.519700 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7fe7-aa24-40ad-8082-d40d77507aca None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:15.520206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7fe7-aa24-40ad-8082-d40d77507aca None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:15.521277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7fe7-aa24-40ad-8082-d40d77507aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:15.613020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-3c6c7fe7-aa24-40ad-8082-d40d77507aca None None] MFA Rules not processed for user `9de538cf1e954ca399379c830a4d44d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:15.614318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3c6c7fe7-aa24-40ad-8082-d40d77507aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:15.753611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de42ff6-5f4c-423c-b7e8-f8b6767c998f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:15.876329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 707/1406] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:15.879364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c7fe7-aa24-40ad-8082-d40d77507aca None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:15.923270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:15.943130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9218d2b-84a3-4aaa-84e8-1598a391fd56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:15.943945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9218d2b-84a3-4aaa-84e8-1598a391fd56 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:15.944369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9218d2b-84a3-4aaa-84e8-1598a391fd56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:15.944667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9218d2b-84a3-4aaa-84e8-1598a391fd56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:16.083595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:16.266809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c7fe7-aa24-40ad-8082-d40d77507aca None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:16.268202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9218d2b-84a3-4aaa-84e8-1598a391fd56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:16.271695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9218d2b-84a3-4aaa-84e8-1598a391fd56 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:16.288505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bdb4e-439b-48d4-9742-95fffc033174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:16.293551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bdb4e-439b-48d4-9742-95fffc033174 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:16.293815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bdb4e-439b-48d4-9742-95fffc033174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:16.294016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bdb4e-439b-48d4-9742-95fffc033174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:16.300577 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bdb4e-439b-48d4-9742-95fffc033174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:16.308840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 700/1407] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 937 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:16.310507 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bdb4e-439b-48d4-9742-95fffc033174 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:16.445948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 708/1408] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:15 2022] POST /identity/v3/projects => generated 399 bytes in 569 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:16.450063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9218d2b-84a3-4aaa-84e8-1598a391fd56 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:16.466317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9218d2b-84a3-4aaa-84e8-1598a391fd56 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:16.841733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed0042-3667-4fc9-9c40-c774c7460cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:16.842429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed0042-3667-4fc9-9c40-c774c7460cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:16.843115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed0042-3667-4fc9-9c40-c774c7460cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:16.843396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed0042-3667-4fc9-9c40-c774c7460cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:16.844041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed0042-3667-4fc9-9c40-c774c7460cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:16.847428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed0042-3667-4fc9-9c40-c774c7460cc0 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:16.921699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 701/1409] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:16 2022] POST /identity/v3/users => generated 306 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:16.926586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bdb4e-439b-48d4-9742-95fffc033174 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:16.949313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bdb4e-439b-48d4-9742-95fffc033174 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:16.976601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf80d06-2c38-43d6-aef4-90005cad5c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:16.977248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf80d06-2c38-43d6-aef4-90005cad5c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:16.977520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf80d06-2c38-43d6-aef4-90005cad5c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:16.977781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf80d06-2c38-43d6-aef4-90005cad5c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:16.978440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf80d06-2c38-43d6-aef4-90005cad5c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:16.980943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf80d06-2c38-43d6-aef4-90005cad5c85 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:17.023086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed0042-3667-4fc9-9c40-c774c7460cc0 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:17.033757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 709/1410] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:17.037787 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed0042-3667-4fc9-9c40-c774c7460cc0 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:17.045704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74876e8-f92a-4a82-ac77-c571d913d8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:17.046452 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74876e8-f92a-4a82-ac77-c571d913d8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:17.046568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74876e8-f92a-4a82-ac77-c571d913d8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:17.046817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74876e8-f92a-4a82-ac77-c571d913d8cd admin admin] PATH_INFO: `/v3/projects/aceec3195a1e49799982e6ea52bfa9b9/users/5d6ef8d8ed194f95a5686e4bd35a3b69/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:17.114614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74876e8-f92a-4a82-ac77-c571d913d8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aceec3195a1e49799982e6ea52bfa9b9, user_id=5d6ef8d8ed194f95a5686e4bd35a3b69, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:17.122288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74876e8-f92a-4a82-ac77-c571d913d8cd admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:17.171331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 702/1411] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:17 2022] PUT /identity/v3/projects/aceec3195a1e49799982e6ea52bfa9b9/users/5d6ef8d8ed194f95a5686e4bd35a3b69/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:17.171477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf80d06-2c38-43d6-aef4-90005cad5c85 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:17.183468 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf80d06-2c38-43d6-aef4-90005cad5c85 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:17.371866 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d9f59-c6e1-46fb-bebb-ca7c64690f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:17.372494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d9f59-c6e1-46fb-bebb-ca7c64690f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:17.372772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d9f59-c6e1-46fb-bebb-ca7c64690f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:17.373040 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d9f59-c6e1-46fb-bebb-ca7c64690f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:17.373695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d9f59-c6e1-46fb-bebb-ca7c64690f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:17.375482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d9f59-c6e1-46fb-bebb-ca7c64690f99 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:17.399899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 710/1412] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:06:17.404925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74876e8-f92a-4a82-ac77-c571d913d8cd admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:17.431996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74876e8-f92a-4a82-ac77-c571d913d8cd admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:17.441614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb4faf-3d81-4ba0-94e4-79e5ea7b1600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:17.442333 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb4faf-3d81-4ba0-94e4-79e5ea7b1600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:17.442616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb4faf-3d81-4ba0-94e4-79e5ea7b1600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:17.442950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb4faf-3d81-4ba0-94e4-79e5ea7b1600 admin admin] PATH_INFO: `/v3/projects/aceec3195a1e49799982e6ea52bfa9b9/users/5d6ef8d8ed194f95a5686e4bd35a3b69/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:17.445384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb4faf-3d81-4ba0-94e4-79e5ea7b1600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aceec3195a1e49799982e6ea52bfa9b9, user_id=5d6ef8d8ed194f95a5686e4bd35a3b69, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:17.448798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb4faf-3d81-4ba0-94e4-79e5ea7b1600 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:17.490924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4414a8d-2ab0-4068-b004-859d9d572262 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:17.491488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4414a8d-2ab0-4068-b004-859d9d572262 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:17.491894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4414a8d-2ab0-4068-b004-859d9d572262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:17.493188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 703/1413] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:17 2022] PUT /identity/v3/projects/aceec3195a1e49799982e6ea52bfa9b9/users/5d6ef8d8ed194f95a5686e4bd35a3b69/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:17.549829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a4414a8d-2ab0-4068-b004-859d9d572262 None None] MFA Rules not processed for user `5d6ef8d8ed194f95a5686e4bd35a3b69`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:17.553768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a4414a8d-2ab0-4068-b004-859d9d572262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:17.710823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 711/1414] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:17 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:17.720089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb4faf-3d81-4ba0-94e4-79e5ea7b1600 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:17.724356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3bbb4faf-3d81-4ba0-94e4-79e5ea7b1600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:17.990688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb4faf-3d81-4ba0-94e4-79e5ea7b1600 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:17.997352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49514ecf-48e2-404c-917c-3fc3714915e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:17.998158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49514ecf-48e2-404c-917c-3fc3714915e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:17.998446 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49514ecf-48e2-404c-917c-3fc3714915e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:17.998716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49514ecf-48e2-404c-917c-3fc3714915e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:18.014088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:18.025285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49514ecf-48e2-404c-917c-3fc3714915e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:18.028092 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49514ecf-48e2-404c-917c-3fc3714915e3 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:18.068596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 704/1415] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:17 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:18.365826 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4414a8d-2ab0-4068-b004-859d9d572262 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:18.367546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a4414a8d-2ab0-4068-b004-859d9d572262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:18.395682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4414a8d-2ab0-4068-b004-859d9d572262 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:18.401241 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5bfa6-cfa5-478f-b285-f9bc317a1262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:18.402218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5bfa6-cfa5-478f-b285-f9bc317a1262 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:18.402725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5bfa6-cfa5-478f-b285-f9bc317a1262 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:18.403268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5bfa6-cfa5-478f-b285-f9bc317a1262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:18.456171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:18.623052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5bfa6-cfa5-478f-b285-f9bc317a1262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:18.629911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5bfa6-cfa5-478f-b285-f9bc317a1262 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:18.658443 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 712/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:18.667709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49514ecf-48e2-404c-917c-3fc3714915e3 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:18.669178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-49514ecf-48e2-404c-917c-3fc3714915e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:18.726745 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a906d-68bd-4314-96da-22c49a0474da None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:18.727299 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a906d-68bd-4314-96da-22c49a0474da None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:18.727573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a906d-68bd-4314-96da-22c49a0474da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:18.833359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-314a906d-68bd-4314-96da-22c49a0474da None None] MFA Rules not processed for user `c8ca34fc5c5743d0b5e3f4f97ae285e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:18.836940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-314a906d-68bd-4314-96da-22c49a0474da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:19.010921 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49514ecf-48e2-404c-917c-3fc3714915e3 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:19.024753 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:19.026303 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:19.028205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:19.028574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] PATH_INFO: `/v3/projects/7f15987bc1e84bc78bfbb63e323f00f9` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:19.045068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f15987bc1e84bc78bfbb63e323f00f9)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:19.054377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:19.098844 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 713/1417] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:19.107034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8616710-a33f-4c70-9432-35e63cacdcf9 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:19.109254 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8616710-a33f-4c70-9432-35e63cacdcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:19.109554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8616710-a33f-4c70-9432-35e63cacdcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:19.181358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f8616710-a33f-4c70-9432-35e63cacdcf9 None None] MFA Rules not processed for user `9cffb662f4104e349c81c578bf66ab7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:19.186925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f8616710-a33f-4c70-9432-35e63cacdcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:19.200909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f15987bc1e84bc78bfbb63e323f00f9', 'request_id': 'req-9188960c-fc87-4c83-b48f-bc046d0c9786'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:19.217693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f15987bc1e84bc78bfbb63e323f00f9', 'request_id': 'req-9188960c-fc87-4c83-b48f-bc046d0c9786'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:19.240813 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f15987bc1e84bc78bfbb63e323f00f9', 'request_id': 'req-9188960c-fc87-4c83-b48f-bc046d0c9786'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:19.269241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] The token cache is being invalidate because project 7f15987bc1e84bc78bfbb63e323f00f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:19.271766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:19.274441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9188960c-fc87-4c83-b48f-bc046d0c9786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:19.281652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 705/1418] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:18 2022] DELETE /identity/v3/projects/7f15987bc1e84bc78bfbb63e323f00f9 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:19.312395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58bb3-c491-4f4a-990b-f114837e8648 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:19.312676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58bb3-c491-4f4a-990b-f114837e8648 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:19.312959 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58bb3-c491-4f4a-990b-f114837e8648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:19.385173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 714/1419] 10.222.0.45 () {58 vars in 1015 bytes} [Fri Oct 28 19:06:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:19.423007 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-87b58bb3-c491-4f4a-990b-f114837e8648 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:19.442192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-87b58bb3-c491-4f4a-990b-f114837e8648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:19.580067 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8616710-a33f-4c70-9432-35e63cacdcf9 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:19.583210 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f8616710-a33f-4c70-9432-35e63cacdcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:19.605035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8616710-a33f-4c70-9432-35e63cacdcf9 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:19.632297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a07e9-a538-4a20-9379-90dbac585e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:19.632932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a07e9-a538-4a20-9379-90dbac585e6d service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:19.633213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a07e9-a538-4a20-9379-90dbac585e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:19.633474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a07e9-a538-4a20-9379-90dbac585e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:19.673396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a07e9-a538-4a20-9379-90dbac585e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:19.675447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a07e9-a538-4a20-9379-90dbac585e6d service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:19.777141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 715/1420] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:19.796975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a07e9-a538-4a20-9379-90dbac585e6d service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:19.798734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-096a07e9-a538-4a20-9379-90dbac585e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:19.812663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 706/1421] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:19.817618 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76439c5f-aaf2-4d7a-b6c8-026c076c866e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:19.817911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76439c5f-aaf2-4d7a-b6c8-026c076c866e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:19.819469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76439c5f-aaf2-4d7a-b6c8-026c076c866e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:19.827806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a07e9-a538-4a20-9379-90dbac585e6d service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:19.842825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:19.843972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:19.844672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:19.847344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] PATH_INFO: `/v3/users/13a8006701a74384aec23f541686b662` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:19.850690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a8006701a74384aec23f541686b662)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:19.865252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:19.941444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-76439c5f-aaf2-4d7a-b6c8-026c076c866e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:19.943077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-76439c5f-aaf2-4d7a-b6c8-026c076c866e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:20.017890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a8006701a74384aec23f541686b662'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:20.018223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 707/1422] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:20.019910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76439c5f-aaf2-4d7a-b6c8-026c076c866e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:20.021042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a8006701a74384aec23f541686b662'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:20.044629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:20.057399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a8006701a74384aec23f541686b662'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:20.061493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76439c5f-aaf2-4d7a-b6c8-026c076c866e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:20.073555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:20.074186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:20.078356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a8006701a74384aec23f541686b662'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:20.079462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:20.080108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:20.080648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf38f3b1-bb1f-4b7e-9fa2-d4662d1fcd8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a8006701a74384aec23f541686b662'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:20.083995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:20.084946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:20.091963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:20.115326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 716/1423] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:19 2022] DELETE /identity/v3/users/13a8006701a74384aec23f541686b662 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:20.119620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e25b9-e1d9-4d23-8398-1a148bc9faa0 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:20.121460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e25b9-e1d9-4d23-8398-1a148bc9faa0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:20.121856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e25b9-e1d9-4d23-8398-1a148bc9faa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:20.147261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 708/1424] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:20.151825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:20.164343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:20.210457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-935e25b9-e1d9-4d23-8398-1a148bc9faa0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:20.211739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-935e25b9-e1d9-4d23-8398-1a148bc9faa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:20.213057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:20.232824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68181e9-2d3f-48a0-8ef7-824d6d4dd79f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:20.395774 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 717/1425] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:20.429192 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e25b9-e1d9-4d23-8398-1a148bc9faa0 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:20.431380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-935e25b9-e1d9-4d23-8398-1a148bc9faa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:20.460266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c584f9b-7fc9-4142-9c0c-f579c0c28e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:20.461211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c584f9b-7fc9-4142-9c0c-f579c0c28e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:20.461211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c584f9b-7fc9-4142-9c0c-f579c0c28e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:20.461501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c584f9b-7fc9-4142-9c0c-f579c0c28e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:20.462141 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c584f9b-7fc9-4142-9c0c-f579c0c28e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:20.464125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c584f9b-7fc9-4142-9c0c-f579c0c28e51 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:20.554493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 709/1426] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:20 2022] POST /identity/v3/projects => generated 363 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:20.558762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c584f9b-7fc9-4142-9c0c-f579c0c28e51 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:20.735733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e25b9-e1d9-4d23-8398-1a148bc9faa0 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:20.795994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c584f9b-7fc9-4142-9c0c-f579c0c28e51 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:20.804798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d72db-fdb5-4f20-874a-eac936f49f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:20.805660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d72db-fdb5-4f20-874a-eac936f49f12 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:20.806120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d72db-fdb5-4f20-874a-eac936f49f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:20.806929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d72db-fdb5-4f20-874a-eac936f49f12 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:20.809442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d72db-fdb5-4f20-874a-eac936f49f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:20.812304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d72db-fdb5-4f20-874a-eac936f49f12 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:20.872683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 710/1427] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:20 2022] POST /identity/v3/users => generated 282 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:20.874801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d72db-fdb5-4f20-874a-eac936f49f12 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:20.877287 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-777d72db-fdb5-4f20-874a-eac936f49f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:20.907495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d72db-fdb5-4f20-874a-eac936f49f12 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:20.914279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:20.915313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:20.915443 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfab95-5eca-4620-b5f0-06619fe07e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:20.915852 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:20.916052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfab95-5eca-4620-b5f0-06619fe07e01 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:20.916684 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfab95-5eca-4620-b5f0-06619fe07e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:20.916883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfab95-5eca-4620-b5f0-06619fe07e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:20.917615 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] PATH_INFO: `/v3/users/42b30bc1e6c14efcb24a7cdbd40be3af` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:20.920325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b30bc1e6c14efcb24a7cdbd40be3af)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:20.922651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:20.930550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfab95-5eca-4620-b5f0-06619fe07e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:20.933350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfab95-5eca-4620-b5f0-06619fe07e01 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:20.961908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 718/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:06:20.963045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfab95-5eca-4620-b5f0-06619fe07e01 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:20.975681 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfab95-5eca-4620-b5f0-06619fe07e01 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:20.982384 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb433d7e-0320-4f20-a789-1bba6c5a6ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:20.983197 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb433d7e-0320-4f20-a789-1bba6c5a6ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:20.983618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb433d7e-0320-4f20-a789-1bba6c5a6ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:20.985654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb433d7e-0320-4f20-a789-1bba6c5a6ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:20.986519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb433d7e-0320-4f20-a789-1bba6c5a6ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:20.992093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb433d7e-0320-4f20-a789-1bba6c5a6ac9 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:21.007296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 719/1429] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:21.016196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb433d7e-0320-4f20-a789-1bba6c5a6ac9 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:21.036569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb433d7e-0320-4f20-a789-1bba6c5a6ac9 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:21.051019 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a948a-e1cb-4303-9809-8a18231728b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:21.051646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a948a-e1cb-4303-9809-8a18231728b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:21.051927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a948a-e1cb-4303-9809-8a18231728b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:21.054503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a948a-e1cb-4303-9809-8a18231728b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:21.055706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a948a-e1cb-4303-9809-8a18231728b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:21.058342 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a948a-e1cb-4303-9809-8a18231728b4 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:21.080932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 720/1430] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:21.082394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a948a-e1cb-4303-9809-8a18231728b4 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:21.091192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b30bc1e6c14efcb24a7cdbd40be3af'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:21.091947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b30bc1e6c14efcb24a7cdbd40be3af'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:21.096847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a948a-e1cb-4303-9809-8a18231728b4 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:21.102827 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baab030-492b-4aab-b5e0-84928a165d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:21.103769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baab030-492b-4aab-b5e0-84928a165d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:21.104120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baab030-492b-4aab-b5e0-84928a165d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:21.104603 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baab030-492b-4aab-b5e0-84928a165d14 admin admin] PATH_INFO: `/v3/projects/88ff14c54e324f7b9414f14e7b038f92/users/7b1b415ccc8f4f9fb0452dcdef7c7caf/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:21.125463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:21.131852 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b30bc1e6c14efcb24a7cdbd40be3af'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:21.146613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b30bc1e6c14efcb24a7cdbd40be3af'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:21.148799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:21.149452 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baab030-492b-4aab-b5e0-84928a165d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ff14c54e324f7b9414f14e7b038f92, user_id=7b1b415ccc8f4f9fb0452dcdef7c7caf, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:21.165541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baab030-492b-4aab-b5e0-84928a165d14 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:21.167534 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-911a2540-ddb8-4bbe-8b49-d9819e8ef7c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b30bc1e6c14efcb24a7cdbd40be3af'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:21.185108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 711/1431] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:20 2022] DELETE /identity/v3/users/42b30bc1e6c14efcb24a7cdbd40be3af => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:21.189433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed815a1-35c7-4adc-b996-44990ff2d1d4 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:21.190167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed815a1-35c7-4adc-b996-44990ff2d1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:21.191484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed815a1-35c7-4adc-b996-44990ff2d1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:21.230539 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 721/1432] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:21 2022] PUT /identity/v3/projects/88ff14c54e324f7b9414f14e7b038f92/users/7b1b415ccc8f4f9fb0452dcdef7c7caf/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:21.245266 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d2e11-6ce6-43b1-9aab-f99725f182df None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:21.245583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d2e11-6ce6-43b1-9aab-f99725f182df None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:21.245809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d2e11-6ce6-43b1-9aab-f99725f182df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:21.263062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-0ed815a1-35c7-4adc-b996-44990ff2d1d4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:21.269546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0ed815a1-35c7-4adc-b996-44990ff2d1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:21.349001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-3b4d2e11-6ce6-43b1-9aab-f99725f182df None None] MFA Rules not processed for user `7b1b415ccc8f4f9fb0452dcdef7c7caf`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:21.349001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3b4d2e11-6ce6-43b1-9aab-f99725f182df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:21.595498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 712/1433] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:21.613268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 722/1434] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:21.616881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed815a1-35c7-4adc-b996-44990ff2d1d4 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:21.618345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0ed815a1-35c7-4adc-b996-44990ff2d1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:21.659493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed815a1-35c7-4adc-b996-44990ff2d1d4 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:21.681147 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88592e-edeb-4c85-af6c-fbb3ea02b20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:21.681689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88592e-edeb-4c85-af6c-fbb3ea02b20e admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:21.682150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88592e-edeb-4c85-af6c-fbb3ea02b20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:21.682262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88592e-edeb-4c85-af6c-fbb3ea02b20e admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:21.682897 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88592e-edeb-4c85-af6c-fbb3ea02b20e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:21.684864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88592e-edeb-4c85-af6c-fbb3ea02b20e admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:21.701137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d2e11-6ce6-43b1-9aab-f99725f182df None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:21.702624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3b4d2e11-6ce6-43b1-9aab-f99725f182df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:21.784338 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 713/1435] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:21 2022] POST /identity/v3/projects => generated 365 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:21.790802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88592e-edeb-4c85-af6c-fbb3ea02b20e admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:21.995940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88592e-edeb-4c85-af6c-fbb3ea02b20e admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:22.012484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd08e7-bc56-4d15-ba0a-cc72314806df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:22.013528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd08e7-bc56-4d15-ba0a-cc72314806df admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:22.013528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd08e7-bc56-4d15-ba0a-cc72314806df admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:22.031189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd08e7-bc56-4d15-ba0a-cc72314806df admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:22.033417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd08e7-bc56-4d15-ba0a-cc72314806df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:22.048232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd08e7-bc56-4d15-ba0a-cc72314806df admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:22.123499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d2e11-6ce6-43b1-9aab-f99725f182df None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:22.170521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 714/1436] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:21 2022] POST /identity/v3/users => generated 289 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:22.172490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd08e7-bc56-4d15-ba0a-cc72314806df admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:22.198245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd08e7-bc56-4d15-ba0a-cc72314806df admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:22.212661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd095e04-1772-448c-ab9a-51e16f091c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:22.214343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd095e04-1772-448c-ab9a-51e16f091c31 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:22.214733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd095e04-1772-448c-ab9a-51e16f091c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:22.215296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd095e04-1772-448c-ab9a-51e16f091c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:22.218134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd095e04-1772-448c-ab9a-51e16f091c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:22.221311 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd095e04-1772-448c-ab9a-51e16f091c31 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:22.251055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 715/1437] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:22.257919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd095e04-1772-448c-ab9a-51e16f091c31 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:22.259203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e3b0-a95c-47d4-81bf-2be9c7c023a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:22.260330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e3b0-a95c-47d4-81bf-2be9c7c023a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:22.265077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e3b0-a95c-47d4-81bf-2be9c7c023a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:22.267227 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e3b0-a95c-47d4-81bf-2be9c7c023a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:22.291364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd095e04-1772-448c-ab9a-51e16f091c31 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:22.294529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96e3b0-a95c-47d4-81bf-2be9c7c023a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:22.298965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96e3b0-a95c-47d4-81bf-2be9c7c023a4 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:22.310295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae63f69-10ac-4720-bd06-0c6626698d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:22.319389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae63f69-10ac-4720-bd06-0c6626698d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:22.319828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae63f69-10ac-4720-bd06-0c6626698d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:22.320200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae63f69-10ac-4720-bd06-0c6626698d70 admin admin] PATH_INFO: `/v3/projects/82a989cfbd2040738ce9fd66ba699f3f/users/ade52bb25a61493aa19a9ebaed696ab9/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:22.358586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 723/1438] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:22.394711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae63f69-10ac-4720-bd06-0c6626698d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a989cfbd2040738ce9fd66ba699f3f, user_id=ade52bb25a61493aa19a9ebaed696ab9, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:22.399828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae63f69-10ac-4720-bd06-0c6626698d70 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:22.451298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 716/1439] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:22 2022] PUT /identity/v3/projects/82a989cfbd2040738ce9fd66ba699f3f/users/ade52bb25a61493aa19a9ebaed696ab9/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:22.456656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e3b0-a95c-47d4-81bf-2be9c7c023a4 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:22.495795 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e3b0-a95c-47d4-81bf-2be9c7c023a4 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:22.628631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:22.713509 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a91797-9e0c-479f-9d90-851862b546a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:22.714363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91797-9e0c-479f-9d90-851862b546a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:22.714468 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91797-9e0c-479f-9d90-851862b546a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:22.714779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91797-9e0c-479f-9d90-851862b546a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:22.715890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a91797-9e0c-479f-9d90-851862b546a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:22.718427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a91797-9e0c-479f-9d90-851862b546a3 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:22.764294 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 724/1440] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:22.765837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae63f69-10ac-4720-bd06-0c6626698d70 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:22.793768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae63f69-10ac-4720-bd06-0c6626698d70 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:22.805881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876c901-8a05-490d-a764-2e97d6814af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:22.806519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876c901-8a05-490d-a764-2e97d6814af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:22.806797 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876c901-8a05-490d-a764-2e97d6814af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:22.807119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876c901-8a05-490d-a764-2e97d6814af4 admin admin] PATH_INFO: `/v3/projects/82a989cfbd2040738ce9fd66ba699f3f/users/ade52bb25a61493aa19a9ebaed696ab9/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:22.820359 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876c901-8a05-490d-a764-2e97d6814af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a989cfbd2040738ce9fd66ba699f3f, user_id=ade52bb25a61493aa19a9ebaed696ab9, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:22.824351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876c901-8a05-490d-a764-2e97d6814af4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:22.870093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 717/1441] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:22 2022] PUT /identity/v3/projects/82a989cfbd2040738ce9fd66ba699f3f/users/ade52bb25a61493aa19a9ebaed696ab9/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:22.871662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097d48d-db53-4c71-8b5c-9ab12153c7bb None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:22.871972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097d48d-db53-4c71-8b5c-9ab12153c7bb None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:22.872218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097d48d-db53-4c71-8b5c-9ab12153c7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:22.912733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-5097d48d-db53-4c71-8b5c-9ab12153c7bb None None] MFA Rules not processed for user `ade52bb25a61493aa19a9ebaed696ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:22.913946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5097d48d-db53-4c71-8b5c-9ab12153c7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:23.150178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 725/1442] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:23.172099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876c901-8a05-490d-a764-2e97d6814af4 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:23.174583 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6876c901-8a05-490d-a764-2e97d6814af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:23.212559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:INSERT=7,keystone:DELETE=1,keystone:SELECT=92 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:23.276418 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097d48d-db53-4c71-8b5c-9ab12153c7bb None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:23.278572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5097d48d-db53-4c71-8b5c-9ab12153c7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:23.410598 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876c901-8a05-490d-a764-2e97d6814af4 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:23.428822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474faf6-f219-492f-94bc-c44fa59cb072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:23.429674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474faf6-f219-492f-94bc-c44fa59cb072 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:23.430104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474faf6-f219-492f-94bc-c44fa59cb072 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:23.430412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474faf6-f219-492f-94bc-c44fa59cb072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:23.468327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474faf6-f219-492f-94bc-c44fa59cb072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:23.470290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474faf6-f219-492f-94bc-c44fa59cb072 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:23.513650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 718/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:23.619370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097d48d-db53-4c71-8b5c-9ab12153c7bb None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:23.691593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:23.696347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:23.698705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:23.699317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] PATH_INFO: `/v3/projects/bc773996e55c425885e129de3fbde25f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:23.704677 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc773996e55c425885e129de3fbde25f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:23.734211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:23.956628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc773996e55c425885e129de3fbde25f', 'request_id': 'req-ac46662b-ed8a-4ccf-a672-fa665eb955d9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:23.974459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc773996e55c425885e129de3fbde25f', 'request_id': 'req-ac46662b-ed8a-4ccf-a672-fa665eb955d9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:24.004267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc773996e55c425885e129de3fbde25f', 'request_id': 'req-ac46662b-ed8a-4ccf-a672-fa665eb955d9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:24.010837 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:24.047375 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] The token cache is being invalidate because project bc773996e55c425885e129de3fbde25f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:24.047747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:24.048669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:24.055433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 726/1444] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:23 2022] DELETE /identity/v3/projects/bc773996e55c425885e129de3fbde25f => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:24.059525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474faf6-f219-492f-94bc-c44fa59cb072 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:24.079377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474faf6-f219-492f-94bc-c44fa59cb072 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:24.221036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:24.222310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:24.222556 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:24.223111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] PATH_INFO: `/v3/users/b2631c7962bc42e7964dc0afecad1566` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:24.224523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2631c7962bc42e7964dc0afecad1566)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:24.228337 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:24.385283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2631c7962bc42e7964dc0afecad1566'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:24.386388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2631c7962bc42e7964dc0afecad1566'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:24.418295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2631c7962bc42e7964dc0afecad1566'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:24.451181 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2631c7962bc42e7964dc0afecad1566'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:24.501528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4bb29c1a-98a2-478a-92de-3fba0d039d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2631c7962bc42e7964dc0afecad1566'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:24.525326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 719/1445] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:24 2022] DELETE /identity/v3/users/b2631c7962bc42e7964dc0afecad1566 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:24.830310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:24.833070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:24.850961 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46662b-ed8a-4ccf-a672-fa665eb955d9 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:25.011973 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:25.012908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:25.013185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:25.013460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] PATH_INFO: `/v3/users/7b1b415ccc8f4f9fb0452dcdef7c7caf` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:25.017120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b1b415ccc8f4f9fb0452dcdef7c7caf)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:25.019840 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:25.109603 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b1b415ccc8f4f9fb0452dcdef7c7caf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:25.110308 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b1b415ccc8f4f9fb0452dcdef7c7caf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:25.126916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b1b415ccc8f4f9fb0452dcdef7c7caf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:25.138477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b1b415ccc8f4f9fb0452dcdef7c7caf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:25.139316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b1b415ccc8f4f9fb0452dcdef7c7caf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:25.197226 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 727/1446] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:24 2022] DELETE /identity/v3/users/7b1b415ccc8f4f9fb0452dcdef7c7caf => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:25.198609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dd67e-6775-4ae6-ac9c-9c8df171d5d3 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:25.198904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dd67e-6775-4ae6-ac9c-9c8df171d5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:25.199209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dd67e-6775-4ae6-ac9c-9c8df171d5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:25.236444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-919dd67e-6775-4ae6-ac9c-9c8df171d5d3 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:25.243012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-919dd67e-6775-4ae6-ac9c-9c8df171d5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:25.331129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:25.420151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 720/1447] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:25.624630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:25.626099 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:25.714810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dd67e-6775-4ae6-ac9c-9c8df171d5d3 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:25.716277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-919dd67e-6775-4ae6-ac9c-9c8df171d5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:25.747260 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dd67e-6775-4ae6-ac9c-9c8df171d5d3 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:25.767740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:25.770125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:25.770588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:25.771298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] PATH_INFO: `/v3/users/9de538cf1e954ca399379c830a4d44d3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:25.774910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de538cf1e954ca399379c830a4d44d3)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:25.779266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:25.822044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374eeb58-dfe2-48c1-9fcf-052bc7de47bc admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:25.842964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed466d-3eb4-4713-9aa9-81dfdee91a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:25.844189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed466d-3eb4-4713-9aa9-81dfdee91a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:25.844539 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed466d-3eb4-4713-9aa9-81dfdee91a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:25.844734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed466d-3eb4-4713-9aa9-81dfdee91a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:25.862693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed466d-3eb4-4713-9aa9-81dfdee91a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:25.864821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed466d-3eb4-4713-9aa9-81dfdee91a39 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:25.884607 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 728/1448] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:25.886072 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed466d-3eb4-4713-9aa9-81dfdee91a39 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:25.888431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d1ed466d-3eb4-4713-9aa9-81dfdee91a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:25.905524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de538cf1e954ca399379c830a4d44d3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:25.906707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed466d-3eb4-4713-9aa9-81dfdee91a39 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:25.907669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de538cf1e954ca399379c830a4d44d3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:25.918084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:25.920411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:25.920800 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:25.921611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] PATH_INFO: `/v3/projects/1fc247cec6b24c849e15ccf1fa480e88` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:25.925021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc247cec6b24c849e15ccf1fa480e88)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:25.929633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:25.946741 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de538cf1e954ca399379c830a4d44d3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:25.953002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:25.970627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de538cf1e954ca399379c830a4d44d3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:25.993871 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c3594d4d-970d-495c-a07d-38085db277a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de538cf1e954ca399379c830a4d44d3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.012047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 721/1449] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:25 2022] DELETE /identity/v3/users/9de538cf1e954ca399379c830a4d44d3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:26.015973 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a197-67f5-4755-9c6f-4deae72c89eb None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:26.016275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a197-67f5-4755-9c6f-4deae72c89eb None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:26.016444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a197-67f5-4755-9c6f-4deae72c89eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:26.074635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc247cec6b24c849e15ccf1fa480e88', 'request_id': 'req-94bdbc30-95b1-4e97-a9dd-0395f901564d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.090038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-06f5a197-67f5-4755-9c6f-4deae72c89eb None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:26.094694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-06f5a197-67f5-4755-9c6f-4deae72c89eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:26.097137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc247cec6b24c849e15ccf1fa480e88', 'request_id': 'req-94bdbc30-95b1-4e97-a9dd-0395f901564d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.117750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc247cec6b24c849e15ccf1fa480e88', 'request_id': 'req-94bdbc30-95b1-4e97-a9dd-0395f901564d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.156962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] The token cache is being invalidate because project 1fc247cec6b24c849e15ccf1fa480e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:26.157307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.159473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.161798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 729/1450] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:25 2022] DELETE /identity/v3/projects/1fc247cec6b24c849e15ccf1fa480e88 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:26.166715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:26.169934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:26.302520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 722/1451] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:26.340259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a197-67f5-4755-9c6f-4deae72c89eb None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:26.341486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdbc30-95b1-4e97-a9dd-0395f901564d admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:26.341782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-06f5a197-67f5-4755-9c6f-4deae72c89eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:26.356910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:26.357576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:26.357870 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:26.358158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] PATH_INFO: `/v3/users/c7bf3084392d4d37ae57747d65e2d287` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:26.361597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7bf3084392d4d37ae57747d65e2d287)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:26.365798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:26.456691 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:26.464368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a197-67f5-4755-9c6f-4deae72c89eb None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:26.470673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfab60-e3e1-4d03-adb5-a113a91072ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:26.472562 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfab60-e3e1-4d03-adb5-a113a91072ab service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:26.473470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfab60-e3e1-4d03-adb5-a113a91072ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:26.473828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfab60-e3e1-4d03-adb5-a113a91072ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:26.490477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7bf3084392d4d37ae57747d65e2d287'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.491371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7bf3084392d4d37ae57747d65e2d287'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.498808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfab60-e3e1-4d03-adb5-a113a91072ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:26.501841 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfab60-e3e1-4d03-adb5-a113a91072ab service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:26.511843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7bf3084392d4d37ae57747d65e2d287'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.524282 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7bf3084392d4d37ae57747d65e2d287'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.525359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-45630c70-a410-44bc-afe4-2983150b2588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7bf3084392d4d37ae57747d65e2d287'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:26.528613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 723/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:26.542968 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 730/1453] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:26 2022] DELETE /identity/v3/users/c7bf3084392d4d37ae57747d65e2d287 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:27.163327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674b1fc-c80b-4f2a-89b9-4440c0295575 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:27.164030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674b1fc-c80b-4f2a-89b9-4440c0295575 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:27.164541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674b1fc-c80b-4f2a-89b9-4440c0295575 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:27.167403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 724/1454] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:06:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:06:27.256705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcb99d-c875-47d5-9874-59ee73502644 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:27.257295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcb99d-c875-47d5-9874-59ee73502644 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:27.257368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcb99d-c875-47d5-9874-59ee73502644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:27.300914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-d8fcb99d-c875-47d5-9874-59ee73502644 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:27.306657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d8fcb99d-c875-47d5-9874-59ee73502644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:27.334047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:27.544207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 731/1455] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:06:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:27.590153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674b1fc-c80b-4f2a-89b9-4440c0295575 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:27.593205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a674b1fc-c80b-4f2a-89b9-4440c0295575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:27.718028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674b1fc-c80b-4f2a-89b9-4440c0295575 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:27.725095 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:27.725997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:27.726478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:27.738092 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] PATH_INFO: `/v3/projects/88ff14c54e324f7b9414f14e7b038f92` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:27.739543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88ff14c54e324f7b9414f14e7b038f92)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:27.742942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:27.902445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88ff14c54e324f7b9414f14e7b038f92', 'request_id': 'req-f8e364bc-674d-403c-9764-41c69aae43c6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:27.912882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88ff14c54e324f7b9414f14e7b038f92', 'request_id': 'req-f8e364bc-674d-403c-9764-41c69aae43c6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:27.943874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88ff14c54e324f7b9414f14e7b038f92', 'request_id': 'req-f8e364bc-674d-403c-9764-41c69aae43c6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:27.978897 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] The token cache is being invalidate because project 88ff14c54e324f7b9414f14e7b038f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:27.990460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:27.992130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:27.993949 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 725/1456] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:27 2022] DELETE /identity/v3/projects/88ff14c54e324f7b9414f14e7b038f92 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:28.001560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcb99d-c875-47d5-9874-59ee73502644 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:28.020357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:28.022935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:28.031162 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcb99d-c875-47d5-9874-59ee73502644 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:28.214890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:28.215953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:28.216437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:28.217106 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] PATH_INFO: `/v3/users/ade52bb25a61493aa19a9ebaed696ab9` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:28.231345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade52bb25a61493aa19a9ebaed696ab9)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:28.235041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:28.257517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e364bc-674d-403c-9764-41c69aae43c6 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:28.270374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:28.271066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:28.271356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:28.271659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:28.277422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:28.308604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:28.310418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:28.336318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 726/1457] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:06:28.355370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade52bb25a61493aa19a9ebaed696ab9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.356251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade52bb25a61493aa19a9ebaed696ab9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.377756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:28.379202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:28.384071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade52bb25a61493aa19a9ebaed696ab9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.397653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:28.402969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade52bb25a61493aa19a9ebaed696ab9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.411767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12a4c4-c91b-43cb-94af-663097fb6b80 service nova] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:28.414600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade52bb25a61493aa19a9ebaed696ab9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.422699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:28.423434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:28.423710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:28.423982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] PATH_INFO: `/v3/projects/0a7fa08049a74d6bbb02f67573575d2f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:28.425678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7fa08049a74d6bbb02f67573575d2f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:28.430130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:28.479147 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 732/1458] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:27 2022] DELETE /identity/v3/users/ade52bb25a61493aa19a9ebaed696ab9 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:28.605865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7fa08049a74d6bbb02f67573575d2f', 'request_id': 'req-9f8ec1b2-df85-4b53-afb5-0ba6df238873'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.628478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7fa08049a74d6bbb02f67573575d2f', 'request_id': 'req-9f8ec1b2-df85-4b53-afb5-0ba6df238873'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.643926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7fa08049a74d6bbb02f67573575d2f', 'request_id': 'req-9f8ec1b2-df85-4b53-afb5-0ba6df238873'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.665489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] The token cache is being invalidate because project 0a7fa08049a74d6bbb02f67573575d2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:28.665803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.666471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:28.671148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 727/1459] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:28 2022] DELETE /identity/v3/projects/0a7fa08049a74d6bbb02f67573575d2f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:28.672347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:28.673809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:28.699701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:28.701226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:28.770133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:28.871404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67a97-819c-4d6f-9177-4facb8274bff admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:28.891803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:28.892481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:28.892768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:28.893824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] PATH_INFO: `/v3/users/5d6ef8d8ed194f95a5686e4bd35a3b69` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:28.895243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d6ef8d8ed194f95a5686e4bd35a3b69)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:28.897296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:28.921763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ec1b2-df85-4b53-afb5-0ba6df238873 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:28.939466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:28.941450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:28.941806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:28.942154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] PATH_INFO: `/v3/projects/dc75e4459537414699b9e1a31e52f36b` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:28.943996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc75e4459537414699b9e1a31e52f36b)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:28.947066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:28.999652 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d6ef8d8ed194f95a5686e4bd35a3b69'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.000473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d6ef8d8ed194f95a5686e4bd35a3b69'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.021792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d6ef8d8ed194f95a5686e4bd35a3b69'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.048364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d6ef8d8ed194f95a5686e4bd35a3b69'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.049167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b9c2d191-eb4b-41df-a810-171ce1921a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d6ef8d8ed194f95a5686e4bd35a3b69'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.061928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc75e4459537414699b9e1a31e52f36b', 'request_id': 'req-58bad701-d4a5-4293-9312-b3fdd00b9647'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.076778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 733/1460] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:28 2022] DELETE /identity/v3/users/5d6ef8d8ed194f95a5686e4bd35a3b69 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:29.081169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc75e4459537414699b9e1a31e52f36b', 'request_id': 'req-58bad701-d4a5-4293-9312-b3fdd00b9647'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.099637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc75e4459537414699b9e1a31e52f36b', 'request_id': 'req-58bad701-d4a5-4293-9312-b3fdd00b9647'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.137637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2c899-a312-4e28-b6b5-278eff9bc710 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.137992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2c899-a312-4e28-b6b5-278eff9bc710 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.138183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2c899-a312-4e28-b6b5-278eff9bc710 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.139965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 734/1461] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:06:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 19:06:29.159108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] The token cache is being invalidate because project dc75e4459537414699b9e1a31e52f36b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:29.159108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.159108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-58bad701-d4a5-4293-9312-b3fdd00b9647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:29.164891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 728/1462] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:28 2022] DELETE /identity/v3/projects/dc75e4459537414699b9e1a31e52f36b => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:29.167070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f64c2-f2b0-4593-a74f-c33b872c2da1 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.167070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f64c2-f2b0-4593-a74f-c33b872c2da1 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.167070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f64c2-f2b0-4593-a74f-c33b872c2da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.181779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe0858-d754-49da-b214-a011276da828 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.182080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe0858-d754-49da-b214-a011276da828 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.182352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe0858-d754-49da-b214-a011276da828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.212055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2e9f64c2-f2b0-4593-a74f-c33b872c2da1 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:29.213664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2e9f64c2-f2b0-4593-a74f-c33b872c2da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:29.213910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3efe0858-d754-49da-b214-a011276da828 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:29.215518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3efe0858-d754-49da-b214-a011276da828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:29.236385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:29.351941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 735/1463] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:29.353874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eabac9-afa9-48db-a7da-da58f6024087 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.354161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eabac9-afa9-48db-a7da-da58f6024087 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.354482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eabac9-afa9-48db-a7da-da58f6024087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.385134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 729/1464] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:29.392125 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-61eabac9-afa9-48db-a7da-da58f6024087 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:29.393953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-61eabac9-afa9-48db-a7da-da58f6024087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:29.416332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 736/1465] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:29.420324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe0858-d754-49da-b214-a011276da828 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:29.434880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe0858-d754-49da-b214-a011276da828 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:29.441887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897809c-1162-47fe-bf41-c0f228cb61d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:29.443289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eabac9-afa9-48db-a7da-da58f6024087 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:29.444851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-61eabac9-afa9-48db-a7da-da58f6024087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:29.450740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897809c-1162-47fe-bf41-c0f228cb61d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.450740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897809c-1162-47fe-bf41-c0f228cb61d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.452600 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897809c-1162-47fe-bf41-c0f228cb61d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.453290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897809c-1162-47fe-bf41-c0f228cb61d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:29.453605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897809c-1162-47fe-bf41-c0f228cb61d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:29.461962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897809c-1162-47fe-bf41-c0f228cb61d8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:29.482360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eabac9-afa9-48db-a7da-da58f6024087 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:29.489805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085de2e-9902-42bf-a917-3b9bf8bb99ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:29.490531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085de2e-9902-42bf-a917-3b9bf8bb99ad service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.490880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085de2e-9902-42bf-a917-3b9bf8bb99ad service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.491295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085de2e-9902-42bf-a917-3b9bf8bb99ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.503846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085de2e-9902-42bf-a917-3b9bf8bb99ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:29.505666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085de2e-9902-42bf-a917-3b9bf8bb99ad service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:29.507115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 730/1466] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:29.511974 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6689a-f8a8-4097-8b1e-028255e6b48d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.512798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6689a-f8a8-4097-8b1e-028255e6b48d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.515299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6689a-f8a8-4097-8b1e-028255e6b48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.530291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 737/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:29.533241 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085de2e-9902-42bf-a917-3b9bf8bb99ad service nova] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:29.559352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085de2e-9902-42bf-a917-3b9bf8bb99ad service nova] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:29.573276 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-91e6689a-f8a8-4097-8b1e-028255e6b48d None None] MFA Rules not processed for user `a25c8936b368477880f429e818a321fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:29.574091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32954d29-c0b8-4389-a5ed-de5662160128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:29.575512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-91e6689a-f8a8-4097-8b1e-028255e6b48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:29.576328 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954d29-c0b8-4389-a5ed-de5662160128 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.576745 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954d29-c0b8-4389-a5ed-de5662160128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.578619 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954d29-c0b8-4389-a5ed-de5662160128 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.579562 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32954d29-c0b8-4389-a5ed-de5662160128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:29.582849 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32954d29-c0b8-4389-a5ed-de5662160128 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:29.641493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 738/1468] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:29 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:29.646382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32954d29-c0b8-4389-a5ed-de5662160128 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:29.656007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:29.691199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 731/1469] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:29.845788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6689a-f8a8-4097-8b1e-028255e6b48d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:29.847694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-91e6689a-f8a8-4097-8b1e-028255e6b48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:29.873059 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6689a-f8a8-4097-8b1e-028255e6b48d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:29.881798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32954d29-c0b8-4389-a5ed-de5662160128 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:29.891282 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:29.891983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.892263 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.892532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] PATH_INFO: `/v3/projects/42c71d69ccff44e4aa9ac54741f61caf` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.894592 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42c71d69ccff44e4aa9ac54741f61caf)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:29.896311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e966462-ff7a-4d3e-832c-653ff5edebcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:29.896780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:29.897066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e966462-ff7a-4d3e-832c-653ff5edebcb admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.897361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e966462-ff7a-4d3e-832c-653ff5edebcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.897656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e966462-ff7a-4d3e-832c-653ff5edebcb admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.898281 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e966462-ff7a-4d3e-832c-653ff5edebcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:29.900744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e966462-ff7a-4d3e-832c-653ff5edebcb admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:29.933950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 739/1470] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:29 2022] POST /identity/v3/users => generated 284 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:06:29.942183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e966462-ff7a-4d3e-832c-653ff5edebcb admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:29.967477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e966462-ff7a-4d3e-832c-653ff5edebcb admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:29.983642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18592c7e-c6ae-44c9-8327-9b9e6fd722f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:29.984321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18592c7e-c6ae-44c9-8327-9b9e6fd722f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:29.984658 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18592c7e-c6ae-44c9-8327-9b9e6fd722f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:29.984939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18592c7e-c6ae-44c9-8327-9b9e6fd722f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:29.987965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18592c7e-c6ae-44c9-8327-9b9e6fd722f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:29.993400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18592c7e-c6ae-44c9-8327-9b9e6fd722f2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.023050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42c71d69ccff44e4aa9ac54741f61caf', 'request_id': 'req-085d1444-90d4-47d0-85f4-9c08487c6591'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:30.026270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 740/1471] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:30.027764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18592c7e-c6ae-44c9-8327-9b9e6fd722f2 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:30.040451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42c71d69ccff44e4aa9ac54741f61caf', 'request_id': 'req-085d1444-90d4-47d0-85f4-9c08487c6591'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:30.055173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18592c7e-c6ae-44c9-8327-9b9e6fd722f2 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:30.068019 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42c71d69ccff44e4aa9ac54741f61caf', 'request_id': 'req-085d1444-90d4-47d0-85f4-9c08487c6591'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:30.089641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] The token cache is being invalidate because project 42c71d69ccff44e4aa9ac54741f61caf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:30.090702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:30.092200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-085d1444-90d4-47d0-85f4-9c08487c6591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:30.097846 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 732/1472] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:29 2022] DELETE /identity/v3/projects/42c71d69ccff44e4aa9ac54741f61caf => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:30.102552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18515f5d-06cf-40b3-a7c1-4b203007b9ce None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.103262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18515f5d-06cf-40b3-a7c1-4b203007b9ce None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.103810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18515f5d-06cf-40b3-a7c1-4b203007b9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.172383 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-18515f5d-06cf-40b3-a7c1-4b203007b9ce None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:30.173687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-18515f5d-06cf-40b3-a7c1-4b203007b9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:30.209065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:30.212696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095b944-ea38-4b73-8ecc-1c1da3c7a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:30.213569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095b944-ea38-4b73-8ecc-1c1da3c7a232 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.214020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095b944-ea38-4b73-8ecc-1c1da3c7a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.214511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095b944-ea38-4b73-8ecc-1c1da3c7a232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.217242 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095b944-ea38-4b73-8ecc-1c1da3c7a232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:30.220566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095b944-ea38-4b73-8ecc-1c1da3c7a232 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.253304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 741/1473] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:30.256441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095b944-ea38-4b73-8ecc-1c1da3c7a232 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:30.259656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4095b944-ea38-4b73-8ecc-1c1da3c7a232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:30.277642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 733/1474] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:30.285575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22b9b9-e695-4772-ba90-428444064796 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.286151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22b9b9-e695-4772-ba90-428444064796 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.286429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22b9b9-e695-4772-ba90-428444064796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.288389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095b944-ea38-4b73-8ecc-1c1da3c7a232 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:30.297239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4db9e1-d008-45ce-9c78-20c887745a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:30.298066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4db9e1-d008-45ce-9c78-20c887745a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.298446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4db9e1-d008-45ce-9c78-20c887745a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.302174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4db9e1-d008-45ce-9c78-20c887745a86 admin admin] PATH_INFO: `/v3/projects/a4221290e7714968889e8772bfa77a7c/users/cf8cd789384b4334ab141b4def1a8a57/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.319259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-ae22b9b9-e695-4772-ba90-428444064796 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:30.321257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-ae22b9b9-e695-4772-ba90-428444064796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:30.324690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4db9e1-d008-45ce-9c78-20c887745a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4221290e7714968889e8772bfa77a7c, user_id=cf8cd789384b4334ab141b4def1a8a57, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:30.328144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4db9e1-d008-45ce-9c78-20c887745a86 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.332744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 734/1475] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:30.338417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22b9b9-e695-4772-ba90-428444064796 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:30.356374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22b9b9-e695-4772-ba90-428444064796 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:30.367215 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 742/1476] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:30 2022] PUT /identity/v3/projects/a4221290e7714968889e8772bfa77a7c/users/cf8cd789384b4334ab141b4def1a8a57/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:30.368853 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72984d-a2f7-4a30-89fb-4743819cb65c None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.369189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72984d-a2f7-4a30-89fb-4743819cb65c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.369470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72984d-a2f7-4a30-89fb-4743819cb65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.414215 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ab72984d-a2f7-4a30-89fb-4743819cb65c None None] MFA Rules not processed for user `cf8cd789384b4334ab141b4def1a8a57`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:30.415563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ab72984d-a2f7-4a30-89fb-4743819cb65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:30.477212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:30.496430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed11a3-3348-455b-8527-9b00d66534ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:30.497108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed11a3-3348-455b-8527-9b00d66534ae admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.497640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed11a3-3348-455b-8527-9b00d66534ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.497980 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed11a3-3348-455b-8527-9b00d66534ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.498735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed11a3-3348-455b-8527-9b00d66534ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:30.499060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed11a3-3348-455b-8527-9b00d66534ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:30.501863 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed11a3-3348-455b-8527-9b00d66534ae admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.511614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 743/1477] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:30.516362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72984d-a2f7-4a30-89fb-4743819cb65c None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:30.528735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 735/1478] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:30.529656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed11a3-3348-455b-8527-9b00d66534ae admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:30.532670 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72984d-a2f7-4a30-89fb-4743819cb65c None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:30.542144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae696a8-c9d0-4448-835b-2146839a0c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:30.543230 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae696a8-c9d0-4448-835b-2146839a0c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.543708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae696a8-c9d0-4448-835b-2146839a0c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.544190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae696a8-c9d0-4448-835b-2146839a0c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.544886 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed11a3-3348-455b-8527-9b00d66534ae admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:30.545466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae696a8-c9d0-4448-835b-2146839a0c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:30.547801 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae696a8-c9d0-4448-835b-2146839a0c3f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.550902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2609bec-78d6-4cb0-9df8-67c00f6f71b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:30.551580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2609bec-78d6-4cb0-9df8-67c00f6f71b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.551840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2609bec-78d6-4cb0-9df8-67c00f6f71b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.552288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2609bec-78d6-4cb0-9df8-67c00f6f71b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.553032 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2609bec-78d6-4cb0-9df8-67c00f6f71b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:30.555352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2609bec-78d6-4cb0-9df8-67c00f6f71b1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.593757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 744/1479] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:30 2022] POST /identity/v3/projects => generated 369 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:06:30.595952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae696a8-c9d0-4448-835b-2146839a0c3f admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:30.675749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 736/1480] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:30 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:30.683607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2609bec-78d6-4cb0-9df8-67c00f6f71b1 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:30.778186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae696a8-c9d0-4448-835b-2146839a0c3f admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:30.816174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2609bec-78d6-4cb0-9df8-67c00f6f71b1 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:30.821211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdd0ed-e485-4b34-8e1b-80a4e688580b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:30.821842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdd0ed-e485-4b34-8e1b-80a4e688580b admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.822126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdd0ed-e485-4b34-8e1b-80a4e688580b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.822408 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdd0ed-e485-4b34-8e1b-80a4e688580b admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.823084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdd0ed-e485-4b34-8e1b-80a4e688580b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:30.825183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdd0ed-e485-4b34-8e1b-80a4e688580b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.851236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 737/1481] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:30 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:30.855457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdd0ed-e485-4b34-8e1b-80a4e688580b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:30.857374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a1bdd0ed-e485-4b34-8e1b-80a4e688580b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:30.867280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c224ace-c144-4ad6-9f31-8a71c70614ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:30.868130 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c224ace-c144-4ad6-9f31-8a71c70614ec admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.868506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c224ace-c144-4ad6-9f31-8a71c70614ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.868867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c224ace-c144-4ad6-9f31-8a71c70614ec admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.869597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c224ace-c144-4ad6-9f31-8a71c70614ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:30.871361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdd0ed-e485-4b34-8e1b-80a4e688580b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:30.871939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c224ace-c144-4ad6-9f31-8a71c70614ec admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.876967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:30.878161 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.878435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.878702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] PATH_INFO: `/v3/projects/aceec3195a1e49799982e6ea52bfa9b9` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.880043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aceec3195a1e49799982e6ea52bfa9b9)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:30.881801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.906228 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 745/1482] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:30 2022] POST /identity/v3/users => generated 291 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:06:30.907339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c224ace-c144-4ad6-9f31-8a71c70614ec admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:30.909354 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5c224ace-c144-4ad6-9f31-8a71c70614ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:30.919305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:30.922620 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c224ace-c144-4ad6-9f31-8a71c70614ec admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:30.929683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:30.930326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:30.930606 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:30.930875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] PATH_INFO: `/v3/projects/82a989cfbd2040738ce9fd66ba699f3f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:30.932239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] RBAC: Authorizing `identity:delete_project(project_id=82a989cfbd2040738ce9fd66ba699f3f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:30.934305 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:30.961557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aceec3195a1e49799982e6ea52bfa9b9', 'request_id': 'req-f4cc70ee-d02a-429a-ba69-8374079cecf8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:30.983523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aceec3195a1e49799982e6ea52bfa9b9', 'request_id': 'req-f4cc70ee-d02a-429a-ba69-8374079cecf8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:30.993505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aceec3195a1e49799982e6ea52bfa9b9', 'request_id': 'req-f4cc70ee-d02a-429a-ba69-8374079cecf8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:31.007127 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] The token cache is being invalidate because project aceec3195a1e49799982e6ea52bfa9b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:31.007419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:31.008142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:31.010917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 738/1483] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:30 2022] DELETE /identity/v3/projects/aceec3195a1e49799982e6ea52bfa9b9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:31.015867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.017857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a989cfbd2040738ce9fd66ba699f3f', 'request_id': 'req-7db28221-d975-4594-97ad-c60bd091942a'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:31.018552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:31.021439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:31.036975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a989cfbd2040738ce9fd66ba699f3f', 'request_id': 'req-7db28221-d975-4594-97ad-c60bd091942a'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:31.046205 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a989cfbd2040738ce9fd66ba699f3f', 'request_id': 'req-7db28221-d975-4594-97ad-c60bd091942a'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:31.060725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] The token cache is being invalidate because project 82a989cfbd2040738ce9fd66ba699f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:31.061017 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:31.063294 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:31.065236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 746/1484] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:30 2022] DELETE /identity/v3/projects/82a989cfbd2040738ce9fd66ba699f3f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:31.065838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.067300 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:31.123409 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc70ee-d02a-429a-ba69-8374079cecf8 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.151161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db28221-d975-4594-97ad-c60bd091942a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.158820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e3817-04e7-4266-84e7-6e6bcbf74861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.159573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e3817-04e7-4266-84e7-6e6bcbf74861 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.159925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e3817-04e7-4266-84e7-6e6bcbf74861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.160878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e3817-04e7-4266-84e7-6e6bcbf74861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.161542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e3817-04e7-4266-84e7-6e6bcbf74861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.163189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e3817-04e7-4266-84e7-6e6bcbf74861 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.177608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 747/1485] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:31.179011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4953-b773-49b2-9d5a-9607d4333aff None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.179595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4953-b773-49b2-9d5a-9607d4333aff None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.179700 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4953-b773-49b2-9d5a-9607d4333aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.208799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1378b9-a1f4-4422-959e-317075e4ad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.209516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1378b9-a1f4-4422-959e-317075e4ad5a admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.209802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1378b9-a1f4-4422-959e-317075e4ad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.210133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1378b9-a1f4-4422-959e-317075e4ad5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.210857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1378b9-a1f4-4422-959e-317075e4ad5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.212938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1378b9-a1f4-4422-959e-317075e4ad5a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.229345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-016b4953-b773-49b2-9d5a-9607d4333aff None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:31.230516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-016b4953-b773-49b2-9d5a-9607d4333aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:31.235664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 739/1486] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:31.237597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cca01-6802-4305-a7cb-e0d4a86cf2a0 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.237887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cca01-6802-4305-a7cb-e0d4a86cf2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.238164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cca01-6802-4305-a7cb-e0d4a86cf2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.248609 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 748/1487] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:31.249549 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4953-b773-49b2-9d5a-9607d4333aff None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.261012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6d7cca01-6802-4305-a7cb-e0d4a86cf2a0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:31.262553 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6d7cca01-6802-4305-a7cb-e0d4a86cf2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:31.262694 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4953-b773-49b2-9d5a-9607d4333aff None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.269221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e76711-3061-43e1-b18d-0dc472eed17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.269893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e76711-3061-43e1-b18d-0dc472eed17d admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.270168 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e76711-3061-43e1-b18d-0dc472eed17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.271030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e76711-3061-43e1-b18d-0dc472eed17d admin admin] PATH_INFO: `/v3/projects/4b0943e586384e1ba7c63555d70686ee/users/07ac544d18974577b47f98ca25822abf/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.273901 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 740/1488] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:31.274922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cca01-6802-4305-a7cb-e0d4a86cf2a0 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.276692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6d7cca01-6802-4305-a7cb-e0d4a86cf2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:31.295324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cca01-6802-4305-a7cb-e0d4a86cf2a0 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.295324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7af764-7ae3-4589-a0e6-1f444dd387a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.295324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af764-7ae3-4589-a0e6-1f444dd387a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.295324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af764-7ae3-4589-a0e6-1f444dd387a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.295628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af764-7ae3-4589-a0e6-1f444dd387a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.295628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7af764-7ae3-4589-a0e6-1f444dd387a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.296894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7af764-7ae3-4589-a0e6-1f444dd387a4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.300405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e76711-3061-43e1-b18d-0dc472eed17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0943e586384e1ba7c63555d70686ee, user_id=07ac544d18974577b47f98ca25822abf, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.303389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e76711-3061-43e1-b18d-0dc472eed17d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.308731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 741/1489] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:31.309099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfd1ca-255f-462c-9a78-ef5ecf1d8842 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.309099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfd1ca-255f-462c-9a78-ef5ecf1d8842 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.309345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfd1ca-255f-462c-9a78-ef5ecf1d8842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.320797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 749/1490] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:31 2022] PUT /identity/v3/projects/4b0943e586384e1ba7c63555d70686ee/users/07ac544d18974577b47f98ca25822abf/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:31.324768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7f1be-fba8-43ad-8247-338329cc3946 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.325062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7f1be-fba8-43ad-8247-338329cc3946 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.325335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7f1be-fba8-43ad-8247-338329cc3946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.342607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f7dfd1ca-255f-462c-9a78-ef5ecf1d8842 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:31.344216 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f7dfd1ca-255f-462c-9a78-ef5ecf1d8842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:31.345573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-88d7f1be-fba8-43ad-8247-338329cc3946 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:31.346789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-88d7f1be-fba8-43ad-8247-338329cc3946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:31.373614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:31.413726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 742/1491] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:31.414430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfd1ca-255f-462c-9a78-ef5ecf1d8842 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.423638 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfd1ca-255f-462c-9a78-ef5ecf1d8842 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.427568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 750/1492] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:31.428263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7f1be-fba8-43ad-8247-338329cc3946 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.432835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9b862-2157-47b3-a4a8-65a2328f08f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.433494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9b862-2157-47b3-a4a8-65a2328f08f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.433771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9b862-2157-47b3-a4a8-65a2328f08f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.434842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9b862-2157-47b3-a4a8-65a2328f08f1 admin admin] PATH_INFO: `/v3/projects/adb7f97c4bfa4323881e08de6d83bc7d/users/0da20a5c53b14b8694167d7180cd8832/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.438138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7f1be-fba8-43ad-8247-338329cc3946 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.444359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034b71-4d9f-4bbd-ae98-bc8ee9f9d462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.445129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034b71-4d9f-4bbd-ae98-bc8ee9f9d462 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.445472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034b71-4d9f-4bbd-ae98-bc8ee9f9d462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.445808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034b71-4d9f-4bbd-ae98-bc8ee9f9d462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.446555 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034b71-4d9f-4bbd-ae98-bc8ee9f9d462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.448431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034b71-4d9f-4bbd-ae98-bc8ee9f9d462 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.452619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9b862-2157-47b3-a4a8-65a2328f08f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adb7f97c4bfa4323881e08de6d83bc7d, user_id=0da20a5c53b14b8694167d7180cd8832, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.455918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9b862-2157-47b3-a4a8-65a2328f08f1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.466247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 751/1493] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:31.466247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034b71-4d9f-4bbd-ae98-bc8ee9f9d462 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.475362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034b71-4d9f-4bbd-ae98-bc8ee9f9d462 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.478743 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 743/1494] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:31 2022] PUT /identity/v3/projects/adb7f97c4bfa4323881e08de6d83bc7d/users/0da20a5c53b14b8694167d7180cd8832/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:31.480298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9b862-2157-47b3-a4a8-65a2328f08f1 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.524695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:31.559362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711776c3-3cc3-4139-8e42-6c749576b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.559805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9b862-2157-47b3-a4a8-65a2328f08f1 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.560145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711776c3-3cc3-4139-8e42-6c749576b228 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.560471 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711776c3-3cc3-4139-8e42-6c749576b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.560885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711776c3-3cc3-4139-8e42-6c749576b228 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.561623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711776c3-3cc3-4139-8e42-6c749576b228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:31.561900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711776c3-3cc3-4139-8e42-6c749576b228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.563640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711776c3-3cc3-4139-8e42-6c749576b228 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.565554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b5685-a3d7-4cd8-abac-82fc5ca2300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.566203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b5685-a3d7-4cd8-abac-82fc5ca2300f admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.566487 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b5685-a3d7-4cd8-abac-82fc5ca2300f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.566813 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b5685-a3d7-4cd8-abac-82fc5ca2300f admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.567571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b5685-a3d7-4cd8-abac-82fc5ca2300f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:31.567967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b5685-a3d7-4cd8-abac-82fc5ca2300f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.569581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b5685-a3d7-4cd8-abac-82fc5ca2300f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.578142 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 752/1495] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:31.582321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711776c3-3cc3-4139-8e42-6c749576b228 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.583066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 744/1496] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:31.584411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff48897-8d53-481c-997e-dc36fcc5ff27 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.584715 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff48897-8d53-481c-997e-dc36fcc5ff27 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.585025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff48897-8d53-481c-997e-dc36fcc5ff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.596611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711776c3-3cc3-4139-8e42-6c749576b228 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.601338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f121cd4-275a-48a1-bc35-887088712e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.602092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f121cd4-275a-48a1-bc35-887088712e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.602446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f121cd4-275a-48a1-bc35-887088712e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.603162 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f121cd4-275a-48a1-bc35-887088712e16 admin admin] PATH_INFO: `/v3/projects/4b0943e586384e1ba7c63555d70686ee/users/07ac544d18974577b47f98ca25822abf/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.605221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f121cd4-275a-48a1-bc35-887088712e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0943e586384e1ba7c63555d70686ee, user_id=07ac544d18974577b47f98ca25822abf, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.607957 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f121cd4-275a-48a1-bc35-887088712e16 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.615222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-eff48897-8d53-481c-997e-dc36fcc5ff27 None None] MFA Rules not processed for user `0da20a5c53b14b8694167d7180cd8832`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:31.616510 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-eff48897-8d53-481c-997e-dc36fcc5ff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:31.620815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 753/1497] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:31 2022] PUT /identity/v3/projects/4b0943e586384e1ba7c63555d70686ee/users/07ac544d18974577b47f98ca25822abf/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:31.623892 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f121cd4-275a-48a1-bc35-887088712e16 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.712768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f121cd4-275a-48a1-bc35-887088712e16 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.717756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 745/1498] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:31.718208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff48897-8d53-481c-997e-dc36fcc5ff27 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.721724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93508e-bc98-46fa-bfd4-b2f0d8a39184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.722460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93508e-bc98-46fa-bfd4-b2f0d8a39184 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.722808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93508e-bc98-46fa-bfd4-b2f0d8a39184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.723166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93508e-bc98-46fa-bfd4-b2f0d8a39184 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.724013 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93508e-bc98-46fa-bfd4-b2f0d8a39184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.726009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93508e-bc98-46fa-bfd4-b2f0d8a39184 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.739967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff48897-8d53-481c-997e-dc36fcc5ff27 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.745434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffed95-6be0-495a-835d-dee037955085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.746093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffed95-6be0-495a-835d-dee037955085 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.746354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffed95-6be0-495a-835d-dee037955085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.746626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffed95-6be0-495a-835d-dee037955085 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.747493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffed95-6be0-495a-835d-dee037955085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.751068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffed95-6be0-495a-835d-dee037955085 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.759731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 754/1499] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:31.765087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4eaf96-a954-49ef-97e5-b68ae17b8113 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.765448 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4eaf96-a954-49ef-97e5-b68ae17b8113 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.766565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4eaf96-a954-49ef-97e5-b68ae17b8113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.800640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 746/1500] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:31.801896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffed95-6be0-495a-835d-dee037955085 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.809921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-7d4eaf96-a954-49ef-97e5-b68ae17b8113 None None] MFA Rules not processed for user `07ac544d18974577b47f98ca25822abf`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:31.811256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7d4eaf96-a954-49ef-97e5-b68ae17b8113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:31.815035 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:31.895958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffed95-6be0-495a-835d-dee037955085 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.903758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f33be-0b2f-4e21-9330-15dd630ff816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.904495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f33be-0b2f-4e21-9330-15dd630ff816 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.905803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f33be-0b2f-4e21-9330-15dd630ff816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.906091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f33be-0b2f-4e21-9330-15dd630ff816 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.906740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f33be-0b2f-4e21-9330-15dd630ff816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.910663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f33be-0b2f-4e21-9330-15dd630ff816 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.935415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 755/1501] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:31.936278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4eaf96-a954-49ef-97e5-b68ae17b8113 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.950943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4eaf96-a954-49ef-97e5-b68ae17b8113 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:31.955865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19300ba4-e3bd-4aef-9baa-d24bebabe082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:31.956484 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19300ba4-e3bd-4aef-9baa-d24bebabe082 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:31.956757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19300ba4-e3bd-4aef-9baa-d24bebabe082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:31.957022 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19300ba4-e3bd-4aef-9baa-d24bebabe082 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:31.957648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19300ba4-e3bd-4aef-9baa-d24bebabe082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:31.959604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19300ba4-e3bd-4aef-9baa-d24bebabe082 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:31.963122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 747/1502] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:31.964009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f33be-0b2f-4e21-9330-15dd630ff816 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.996001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 756/1503] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/users => generated 291 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:31.996001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19300ba4-e3bd-4aef-9baa-d24bebabe082 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:31.996001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-19300ba4-e3bd-4aef-9baa-d24bebabe082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:32.009474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:32.097835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f33be-0b2f-4e21-9330-15dd630ff816 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.108761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3613558-1f95-4f84-81c0-94acfb29e049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.109395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3613558-1f95-4f84-81c0-94acfb29e049 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.109681 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3613558-1f95-4f84-81c0-94acfb29e049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.109958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3613558-1f95-4f84-81c0-94acfb29e049 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.111216 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3613558-1f95-4f84-81c0-94acfb29e049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.113144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3613558-1f95-4f84-81c0-94acfb29e049 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.153702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19300ba4-e3bd-4aef-9baa-d24bebabe082 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.167694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 748/1504] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:31 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:32.169772 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3613558-1f95-4f84-81c0-94acfb29e049 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.212887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3613558-1f95-4f84-81c0-94acfb29e049 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.220218 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445dda9-9105-4d8b-8229-2b8f68e51bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.221271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445dda9-9105-4d8b-8229-2b8f68e51bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.221745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445dda9-9105-4d8b-8229-2b8f68e51bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.222223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445dda9-9105-4d8b-8229-2b8f68e51bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.223549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445dda9-9105-4d8b-8229-2b8f68e51bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.225167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445dda9-9105-4d8b-8229-2b8f68e51bf1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.275004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 749/1505] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:32 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:32.275004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445dda9-9105-4d8b-8229-2b8f68e51bf1 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.275407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66d5fe-849c-4ad4-92c4-8da8973b4db5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.275407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66d5fe-849c-4ad4-92c4-8da8973b4db5 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.275407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66d5fe-849c-4ad4-92c4-8da8973b4db5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.275407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66d5fe-849c-4ad4-92c4-8da8973b4db5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.283360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445dda9-9105-4d8b-8229-2b8f68e51bf1 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.290857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098aba1-ba4c-420a-812c-5e1bbbdda98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.291985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098aba1-ba4c-420a-812c-5e1bbbdda98a admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.292490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098aba1-ba4c-420a-812c-5e1bbbdda98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.292976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098aba1-ba4c-420a-812c-5e1bbbdda98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.294040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098aba1-ba4c-420a-812c-5e1bbbdda98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.296093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098aba1-ba4c-420a-812c-5e1bbbdda98a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.315147 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 750/1506] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:32.316361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098aba1-ba4c-420a-812c-5e1bbbdda98a admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.328087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098aba1-ba4c-420a-812c-5e1bbbdda98a admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.333534 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b36ee-c4b0-4ef4-af94-69796aa14531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.334372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b36ee-c4b0-4ef4-af94-69796aa14531 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.337735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b36ee-c4b0-4ef4-af94-69796aa14531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.338122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b36ee-c4b0-4ef4-af94-69796aa14531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.339071 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b36ee-c4b0-4ef4-af94-69796aa14531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.341093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b36ee-c4b0-4ef4-af94-69796aa14531 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.353533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 751/1507] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:32.354304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b36ee-c4b0-4ef4-af94-69796aa14531 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.367970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b36ee-c4b0-4ef4-af94-69796aa14531 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.374643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ff242-89e1-46fc-8aa2-785d543a6e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.375295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ff242-89e1-46fc-8aa2-785d543a6e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.375571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ff242-89e1-46fc-8aa2-785d543a6e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.375842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ff242-89e1-46fc-8aa2-785d543a6e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.376484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ff242-89e1-46fc-8aa2-785d543a6e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.378832 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ff242-89e1-46fc-8aa2-785d543a6e97 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.383448 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66d5fe-849c-4ad4-92c4-8da8973b4db5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.386147 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66d5fe-849c-4ad4-92c4-8da8973b4db5 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.391406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 752/1508] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:32.391996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ff242-89e1-46fc-8aa2-785d543a6e97 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.402456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 757/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:32.407697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ff242-89e1-46fc-8aa2-785d543a6e97 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.413540 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f785c-1fe4-4bf9-a3a6-cd0c9afa363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.414150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f785c-1fe4-4bf9-a3a6-cd0c9afa363d admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.414430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f785c-1fe4-4bf9-a3a6-cd0c9afa363d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.414697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f785c-1fe4-4bf9-a3a6-cd0c9afa363d admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.415374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f785c-1fe4-4bf9-a3a6-cd0c9afa363d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.416079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66d5fe-849c-4ad4-92c4-8da8973b4db5 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.416952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f785c-1fe4-4bf9-a3a6-cd0c9afa363d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.436798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 753/1510] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:32.437708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f785c-1fe4-4bf9-a3a6-cd0c9afa363d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.442445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66d5fe-849c-4ad4-92c4-8da8973b4db5 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.455682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5487b-9e22-4ddb-8077-ed185460d4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.456849 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5487b-9e22-4ddb-8077-ed185460d4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.457259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5487b-9e22-4ddb-8077-ed185460d4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.457610 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5487b-9e22-4ddb-8077-ed185460d4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.458555 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5487b-9e22-4ddb-8077-ed185460d4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.462794 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f785c-1fe4-4bf9-a3a6-cd0c9afa363d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.464450 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5487b-9e22-4ddb-8077-ed185460d4f2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.470650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7853db-4823-42a7-b217-eb991936c78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.472239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7853db-4823-42a7-b217-eb991936c78d admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.473073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7853db-4823-42a7-b217-eb991936c78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.473654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7853db-4823-42a7-b217-eb991936c78d admin admin] PATH_INFO: `/v3/projects/0ad0bc549fb74104ae7d79db2388017e/users/abed80d0ed3f450a94bb7bd993de7c90/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.482376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 758/1511] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:32.483254 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5487b-9e22-4ddb-8077-ed185460d4f2 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.496587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5487b-9e22-4ddb-8077-ed185460d4f2 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.504429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7853db-4823-42a7-b217-eb991936c78d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad0bc549fb74104ae7d79db2388017e, user_id=abed80d0ed3f450a94bb7bd993de7c90, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.505965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357aff8-89ae-4a15-94c8-34d17cd33fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.507199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357aff8-89ae-4a15-94c8-34d17cd33fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.507321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357aff8-89ae-4a15-94c8-34d17cd33fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.508113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357aff8-89ae-4a15-94c8-34d17cd33fb7 admin admin] PATH_INFO: `/v3/projects/a2ec8c0835b84fc3bf0da9af314da8eb/users/6a3a54b717664e0ea3308fb57ae0272f/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.523824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7853db-4823-42a7-b217-eb991936c78d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.534665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:32.554262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357aff8-89ae-4a15-94c8-34d17cd33fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ec8c0835b84fc3bf0da9af314da8eb, user_id=6a3a54b717664e0ea3308fb57ae0272f, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.558768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357aff8-89ae-4a15-94c8-34d17cd33fb7 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.566297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 754/1512] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:32 2022] PUT /identity/v3/projects/0ad0bc549fb74104ae7d79db2388017e/users/abed80d0ed3f450a94bb7bd993de7c90/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:32.566297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7853db-4823-42a7-b217-eb991936c78d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.607079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 759/1513] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:32 2022] PUT /identity/v3/projects/a2ec8c0835b84fc3bf0da9af314da8eb/users/6a3a54b717664e0ea3308fb57ae0272f/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:32.608062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357aff8-89ae-4a15-94c8-34d17cd33fb7 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.669860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:32.685178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7853db-4823-42a7-b217-eb991936c78d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.702238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357aff8-89ae-4a15-94c8-34d17cd33fb7 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.708407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ece5e-220f-4660-8162-63ef5ceb3d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.709369 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ece5e-220f-4660-8162-63ef5ceb3d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.709802 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ece5e-220f-4660-8162-63ef5ceb3d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.710461 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ece5e-220f-4660-8162-63ef5ceb3d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.711499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ece5e-220f-4660-8162-63ef5ceb3d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.713381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ece5e-220f-4660-8162-63ef5ceb3d68 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.725221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 760/1514] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:32.726605 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a45185-179e-476d-800b-2d688fb727b4 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.726894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a45185-179e-476d-800b-2d688fb727b4 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.727349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a45185-179e-476d-800b-2d688fb727b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.763211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafe2ef-aa06-496a-87c5-0dd32db7689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.763916 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafe2ef-aa06-496a-87c5-0dd32db7689f admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.764217 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafe2ef-aa06-496a-87c5-0dd32db7689f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.764478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafe2ef-aa06-496a-87c5-0dd32db7689f admin admin] PATH_INFO: `/v3/projects/0b085d7cb8b2469ebb367df3049c3409/users/30e6afd91dd94ad58ba94ae642149dae/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.783830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafe2ef-aa06-496a-87c5-0dd32db7689f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b085d7cb8b2469ebb367df3049c3409, user_id=30e6afd91dd94ad58ba94ae642149dae, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.786729 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafe2ef-aa06-496a-87c5-0dd32db7689f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.789656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-d9a45185-179e-476d-800b-2d688fb727b4 None None] MFA Rules not processed for user `6a3a54b717664e0ea3308fb57ae0272f`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:32.790952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d9a45185-179e-476d-800b-2d688fb727b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:32.816114 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 755/1515] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:32 2022] PUT /identity/v3/projects/0b085d7cb8b2469ebb367df3049c3409/users/30e6afd91dd94ad58ba94ae642149dae/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:32.817081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafe2ef-aa06-496a-87c5-0dd32db7689f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:32.885242 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 761/1516] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:32.887601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9179eae-3754-4dad-b800-3e09d469ab47 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.887939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9179eae-3754-4dad-b800-3e09d469ab47 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.888273 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9179eae-3754-4dad-b800-3e09d469ab47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.920466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafe2ef-aa06-496a-87c5-0dd32db7689f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:32.930821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f9179eae-3754-4dad-b800-3e09d469ab47 None None] MFA Rules not processed for user `30e6afd91dd94ad58ba94ae642149dae`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:32.932282 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f9179eae-3754-4dad-b800-3e09d469ab47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:32.940642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6d4cf-ab1c-4c57-a37b-bccd42c12912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:32.941571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6d4cf-ab1c-4c57-a37b-bccd42c12912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:32.941979 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6d4cf-ab1c-4c57-a37b-bccd42c12912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:32.942326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6d4cf-ab1c-4c57-a37b-bccd42c12912 admin admin] PATH_INFO: `/v3/projects/0ad0bc549fb74104ae7d79db2388017e/users/abed80d0ed3f450a94bb7bd993de7c90/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:32.950630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6d4cf-ab1c-4c57-a37b-bccd42c12912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad0bc549fb74104ae7d79db2388017e, user_id=abed80d0ed3f450a94bb7bd993de7c90, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:32.953857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6d4cf-ab1c-4c57-a37b-bccd42c12912 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:32.990428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 756/1517] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:32 2022] PUT /identity/v3/projects/0ad0bc549fb74104ae7d79db2388017e/users/abed80d0ed3f450a94bb7bd993de7c90/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:32.994183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6d4cf-ab1c-4c57-a37b-bccd42c12912 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.026689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:33.093771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 762/1518] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:33.095278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34c58a-a0a2-4c93-8a30-87abb2c37a1a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.095695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34c58a-a0a2-4c93-8a30-87abb2c37a1a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.096725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34c58a-a0a2-4c93-8a30-87abb2c37a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.134663 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8a34c58a-a0a2-4c93-8a30-87abb2c37a1a None None] MFA Rules not processed for user `abed80d0ed3f450a94bb7bd993de7c90`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:33.136072 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8a34c58a-a0a2-4c93-8a30-87abb2c37a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:33.179427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6d4cf-ab1c-4c57-a37b-bccd42c12912 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:33.193956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0012-27e7-4739-bb8e-a5b46f01811b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:33.194589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0012-27e7-4739-bb8e-a5b46f01811b admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.194879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0012-27e7-4739-bb8e-a5b46f01811b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.195202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0012-27e7-4739-bb8e-a5b46f01811b admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.195844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f0012-27e7-4739-bb8e-a5b46f01811b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:33.200307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f0012-27e7-4739-bb8e-a5b46f01811b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:33.254041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:33.263328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 757/1519] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:32 2022] POST /identity/v3/projects => generated 383 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:33.263328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0012-27e7-4739-bb8e-a5b46f01811b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.319865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 763/1520] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:33.321117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34c58a-a0a2-4c93-8a30-87abb2c37a1a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.338814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34c58a-a0a2-4c93-8a30-87abb2c37a1a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:33.403561 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0012-27e7-4739-bb8e-a5b46f01811b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:33.425872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb773d64-56b5-416e-9327-2f81d8a7e634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:33.426390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb773d64-56b5-416e-9327-2f81d8a7e634 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.426724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb773d64-56b5-416e-9327-2f81d8a7e634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.426930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb773d64-56b5-416e-9327-2f81d8a7e634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.427588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb773d64-56b5-416e-9327-2f81d8a7e634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:33.429502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb773d64-56b5-416e-9327-2f81d8a7e634 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:33.477323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148f23d-f72a-4912-b938-f61ab647f63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:33.480402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148f23d-f72a-4912-b938-f61ab647f63f admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.480696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148f23d-f72a-4912-b938-f61ab647f63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.480971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148f23d-f72a-4912-b938-f61ab647f63f admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.481614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148f23d-f72a-4912-b938-f61ab647f63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:33.485231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 758/1521] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:33 2022] POST /identity/v3/projects => generated 375 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:33.485386 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148f23d-f72a-4912-b938-f61ab647f63f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:33.487706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb773d64-56b5-416e-9327-2f81d8a7e634 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.489804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-eb773d64-56b5-416e-9327-2f81d8a7e634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:33.530297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 764/1522] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:33 2022] POST /identity/v3/users => generated 298 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:33.532900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148f23d-f72a-4912-b938-f61ab647f63f admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.562110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148f23d-f72a-4912-b938-f61ab647f63f admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:33.611620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb773d64-56b5-416e-9327-2f81d8a7e634 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:33.621503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5617a-3171-4444-b4c5-2999a600c9cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:33.622129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5617a-3171-4444-b4c5-2999a600c9cf service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.622412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5617a-3171-4444-b4c5-2999a600c9cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.622682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5617a-3171-4444-b4c5-2999a600c9cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.681941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c864b-126a-4377-a48f-ba1695d6c724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:33.683310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c864b-126a-4377-a48f-ba1695d6c724 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.683941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c864b-126a-4377-a48f-ba1695d6c724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.684537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c864b-126a-4377-a48f-ba1695d6c724 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.685491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c864b-126a-4377-a48f-ba1695d6c724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:33.690556 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c864b-126a-4377-a48f-ba1695d6c724 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:33.710922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:33.745018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 765/1523] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:33 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:33.747059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c864b-126a-4377-a48f-ba1695d6c724 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.747691 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5617a-3171-4444-b4c5-2999a600c9cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:33.750421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5617a-3171-4444-b4c5-2999a600c9cf service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:33.775622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c864b-126a-4377-a48f-ba1695d6c724 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:33.780991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 759/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:33.782955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5617a-3171-4444-b4c5-2999a600c9cf service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.794594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841d61b-2c4e-448e-a924-0ca744d81003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:33.795283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841d61b-2c4e-448e-a924-0ca744d81003 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.795564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841d61b-2c4e-448e-a924-0ca744d81003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.795831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841d61b-2c4e-448e-a924-0ca744d81003 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.796489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841d61b-2c4e-448e-a924-0ca744d81003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:33.798091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841d61b-2c4e-448e-a924-0ca744d81003 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:33.829775 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5617a-3171-4444-b4c5-2999a600c9cf service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:33.834460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 766/1525] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:33.836330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841d61b-2c4e-448e-a924-0ca744d81003 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.840045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee6fb3-6c9a-48de-8361-a6637d8c6dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:33.841236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee6fb3-6c9a-48de-8361-a6637d8c6dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.841667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee6fb3-6c9a-48de-8361-a6637d8c6dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.842396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee6fb3-6c9a-48de-8361-a6637d8c6dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.843324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee6fb3-6c9a-48de-8361-a6637d8c6dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:33.847666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee6fb3-6c9a-48de-8361-a6637d8c6dd0 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:33.849202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841d61b-2c4e-448e-a924-0ca744d81003 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:33.855100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba81cb-09e3-4864-a0cb-26b223f3d078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:33.855786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba81cb-09e3-4864-a0cb-26b223f3d078 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.856070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba81cb-09e3-4864-a0cb-26b223f3d078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.857252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba81cb-09e3-4864-a0cb-26b223f3d078 admin admin] PATH_INFO: `/v3/projects/8481d4b15a834f67bbe73e991293df0a/users/7efd22c913ac4e8381ad46981c8956f6/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.868215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 760/1526] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:33.873980 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee6fb3-6c9a-48de-8361-a6637d8c6dd0 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.886744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee6fb3-6c9a-48de-8361-a6637d8c6dd0 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:33.900525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983c5bb-bafe-4ea7-8c78-0ea10ecf2799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:33.901216 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983c5bb-bafe-4ea7-8c78-0ea10ecf2799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:33.903843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983c5bb-bafe-4ea7-8c78-0ea10ecf2799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:33.904299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983c5bb-bafe-4ea7-8c78-0ea10ecf2799 admin admin] PATH_INFO: `/v3/projects/fbab451d31a94a239b3e442d8724d9e7/users/8684018a3cae42499c2d607c92e42bf3/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:33.916711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba81cb-09e3-4864-a0cb-26b223f3d078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8481d4b15a834f67bbe73e991293df0a, user_id=7efd22c913ac4e8381ad46981c8956f6, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:33.919401 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba81cb-09e3-4864-a0cb-26b223f3d078 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:33.945742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983c5bb-bafe-4ea7-8c78-0ea10ecf2799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbab451d31a94a239b3e442d8724d9e7, user_id=8684018a3cae42499c2d607c92e42bf3, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:33.952601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983c5bb-bafe-4ea7-8c78-0ea10ecf2799 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:33.960896 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 767/1527] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:33 2022] PUT /identity/v3/projects/8481d4b15a834f67bbe73e991293df0a/users/7efd22c913ac4e8381ad46981c8956f6/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:33.960896 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba81cb-09e3-4864-a0cb-26b223f3d078 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:33.984584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 761/1528] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:33 2022] PUT /identity/v3/projects/fbab451d31a94a239b3e442d8724d9e7/users/8684018a3cae42499c2d607c92e42bf3/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:33.987912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983c5bb-bafe-4ea7-8c78-0ea10ecf2799 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:34.034079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:34.109216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba81cb-09e3-4864-a0cb-26b223f3d078 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:34.162379 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983c5bb-bafe-4ea7-8c78-0ea10ecf2799 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:34.176884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955f615-39c2-460e-ba19-7ca40e2ae508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:34.177514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955f615-39c2-460e-ba19-7ca40e2ae508 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:34.177791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955f615-39c2-460e-ba19-7ca40e2ae508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:34.178059 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955f615-39c2-460e-ba19-7ca40e2ae508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:34.182219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955f615-39c2-460e-ba19-7ca40e2ae508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:34.186450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955f615-39c2-460e-ba19-7ca40e2ae508 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:34.210382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 762/1529] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:34.211271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955f615-39c2-460e-ba19-7ca40e2ae508 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:34.253929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955f615-39c2-460e-ba19-7ca40e2ae508 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:34.268520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:34.271234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:34.271234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:34.271234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] PATH_INFO: `/v3/users/0da20a5c53b14b8694167d7180cd8832` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:34.273176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da20a5c53b14b8694167d7180cd8832)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:34.276045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:34.283367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da49c32-cf10-45cb-b0d0-b5e3741056a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:34.284144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49c32-cf10-45cb-b0d0-b5e3741056a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:34.284431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49c32-cf10-45cb-b0d0-b5e3741056a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:34.284709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49c32-cf10-45cb-b0d0-b5e3741056a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:34.288789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da49c32-cf10-45cb-b0d0-b5e3741056a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:34.290695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da49c32-cf10-45cb-b0d0-b5e3741056a2 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:34.346527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 763/1530] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:34.347966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da49c32-cf10-45cb-b0d0-b5e3741056a2 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:34.368291 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da49c32-cf10-45cb-b0d0-b5e3741056a2 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:34.374426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee600e4a-f641-4bca-994a-b67d77ca2284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:34.375279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee600e4a-f641-4bca-994a-b67d77ca2284 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:34.375771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee600e4a-f641-4bca-994a-b67d77ca2284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:34.376043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee600e4a-f641-4bca-994a-b67d77ca2284 admin admin] PATH_INFO: `/v3/projects/8481d4b15a834f67bbe73e991293df0a/users/7efd22c913ac4e8381ad46981c8956f6/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:34.379466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee600e4a-f641-4bca-994a-b67d77ca2284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8481d4b15a834f67bbe73e991293df0a, user_id=7efd22c913ac4e8381ad46981c8956f6, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:34.382556 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee600e4a-f641-4bca-994a-b67d77ca2284 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:34.399939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 764/1531] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:34 2022] PUT /identity/v3/projects/8481d4b15a834f67bbe73e991293df0a/users/7efd22c913ac4e8381ad46981c8956f6/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:34.400913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee600e4a-f641-4bca-994a-b67d77ca2284 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:34.409370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da20a5c53b14b8694167d7180cd8832'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:34.410441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da20a5c53b14b8694167d7180cd8832'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:34.440353 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da20a5c53b14b8694167d7180cd8832'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:34.456527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da20a5c53b14b8694167d7180cd8832'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:34.457471 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1bf88148-bfc8-4fb6-9cbb-da7401b3944a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da20a5c53b14b8694167d7180cd8832'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:34.479923 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 768/1532] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:33 2022] DELETE /identity/v3/users/0da20a5c53b14b8694167d7180cd8832 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:34.481565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80913d70-0303-4702-a999-c6222bacd2a7 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:34.481684 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee600e4a-f641-4bca-994a-b67d77ca2284 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:34.482050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80913d70-0303-4702-a999-c6222bacd2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:34.483312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80913d70-0303-4702-a999-c6222bacd2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:34.507791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:34.555725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b0000-d93d-437d-9730-db375bbf5e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:34.556344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b0000-d93d-437d-9730-db375bbf5e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:34.556617 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b0000-d93d-437d-9730-db375bbf5e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:34.556890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b0000-d93d-437d-9730-db375bbf5e30 admin admin] PATH_INFO: `/v3/projects/fbab451d31a94a239b3e442d8724d9e7/users/8684018a3cae42499c2d607c92e42bf3/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:34.559030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b0000-d93d-437d-9730-db375bbf5e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbab451d31a94a239b3e442d8724d9e7, user_id=8684018a3cae42499c2d607c92e42bf3, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:34.561768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b0000-d93d-437d-9730-db375bbf5e30 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:34.562830 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-80913d70-0303-4702-a999-c6222bacd2a7 None None] MFA Rules not processed for user `7efd22c913ac4e8381ad46981c8956f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:34.567507 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-80913d70-0303-4702-a999-c6222bacd2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:34.585940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 765/1533] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:34 2022] PUT /identity/v3/projects/fbab451d31a94a239b3e442d8724d9e7/users/8684018a3cae42499c2d607c92e42bf3/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:34.585940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ef83d-fe28-4d27-8a45-bae47efe038e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:34.585940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ef83d-fe28-4d27-8a45-bae47efe038e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:34.585940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ef83d-fe28-4d27-8a45-bae47efe038e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:34.601268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-155ef83d-fe28-4d27-8a45-bae47efe038e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:34.601268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-155ef83d-fe28-4d27-8a45-bae47efe038e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:34.695216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:34.750153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 766/1534] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:34.752315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dfe66-21d1-4ccc-9c2b-ec7fe84eff7c None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:34.752315 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dfe66-21d1-4ccc-9c2b-ec7fe84eff7c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:34.752521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dfe66-21d1-4ccc-9c2b-ec7fe84eff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:34.793523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-cb3dfe66-21d1-4ccc-9c2b-ec7fe84eff7c None None] MFA Rules not processed for user `8684018a3cae42499c2d607c92e42bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:34.794973 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-cb3dfe66-21d1-4ccc-9c2b-ec7fe84eff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:34.822349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 769/1535] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:34.822349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80913d70-0303-4702-a999-c6222bacd2a7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:34.842053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-80913d70-0303-4702-a999-c6222bacd2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:34.931169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 767/1536] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:34.939516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dfe66-21d1-4ccc-9c2b-ec7fe84eff7c None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:34.954388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-cb3dfe66-21d1-4ccc-9c2b-ec7fe84eff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:34.997052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80913d70-0303-4702-a999-c6222bacd2a7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:35.001129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b78662-4b9d-4cff-b02e-0ad7416cd9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:35.001745 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b78662-4b9d-4cff-b02e-0ad7416cd9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:35.002070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b78662-4b9d-4cff-b02e-0ad7416cd9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:35.002625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b78662-4b9d-4cff-b02e-0ad7416cd9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:35.017082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b78662-4b9d-4cff-b02e-0ad7416cd9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:35.018563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b78662-4b9d-4cff-b02e-0ad7416cd9a6 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:35.040107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 770/1537] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:35.040980 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b78662-4b9d-4cff-b02e-0ad7416cd9a6 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:35.045597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c1b78662-4b9d-4cff-b02e-0ad7416cd9a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:35.088059 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:35.115122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dfe66-21d1-4ccc-9c2b-ec7fe84eff7c None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:35.129679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dcdc8-36db-4ff8-a995-f916c1913d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:35.130441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dcdc8-36db-4ff8-a995-f916c1913d73 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:35.130718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dcdc8-36db-4ff8-a995-f916c1913d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:35.131006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dcdc8-36db-4ff8-a995-f916c1913d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:35.155898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dcdc8-36db-4ff8-a995-f916c1913d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:35.163202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dcdc8-36db-4ff8-a995-f916c1913d73 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:35.172002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b78662-4b9d-4cff-b02e-0ad7416cd9a6 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:35.176491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:35.177112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:35.178000 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:35.178284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:35.181086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:35.190544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 768/1538] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:35.336362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:35.337809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:35.372517 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 771/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:35.435210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dcdc8-36db-4ff8-a995-f916c1913d73 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:35.446509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dcdc8-36db-4ff8-a995-f916c1913d73 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:35.455838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33891177-119a-46aa-84d5-e93fcfbc6560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:35.456596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891177-119a-46aa-84d5-e93fcfbc6560 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:35.456878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891177-119a-46aa-84d5-e93fcfbc6560 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:35.457170 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891177-119a-46aa-84d5-e93fcfbc6560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:35.462271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-33891177-119a-46aa-84d5-e93fcfbc6560 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:35.593278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33891177-119a-46aa-84d5-e93fcfbc6560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:35.597329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33891177-119a-46aa-84d5-e93fcfbc6560 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:35.621122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 769/1540] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:35.704726 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:35.706857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:35.751139 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7482d17-7a08-4947-98a1-ccb9836d9c03 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:35.766535 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:35.769038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:35.770051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:35.770533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] PATH_INFO: `/v3/users/cf8cd789384b4334ab141b4def1a8a57` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:35.775450 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf8cd789384b4334ab141b4def1a8a57)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:35.789682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:35.850909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:35.897872 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf8cd789384b4334ab141b4def1a8a57'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:35.899446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf8cd789384b4334ab141b4def1a8a57'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:35.929288 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf8cd789384b4334ab141b4def1a8a57'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:35.949693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf8cd789384b4334ab141b4def1a8a57'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:35.950590 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e472753-0906-445c-b361-70791c1ba64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf8cd789384b4334ab141b4def1a8a57'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:35.985676 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 772/1541] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:35 2022] DELETE /identity/v3/users/cf8cd789384b4334ab141b4def1a8a57 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:35.991057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd62ef-3fb0-4b25-962c-91eea4e3317f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:35.992175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd62ef-3fb0-4b25-962c-91eea4e3317f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:35.992453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd62ef-3fb0-4b25-962c-91eea4e3317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:36.055533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6ccd62ef-3fb0-4b25-962c-91eea4e3317f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:36.059502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6ccd62ef-3fb0-4b25-962c-91eea4e3317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:36.114443 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bd990-e675-4994-8660-7043e6758570 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:36.114821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bd990-e675-4994-8660-7043e6758570 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:36.115339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bd990-e675-4994-8660-7043e6758570 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:36.117515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 773/1542] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:06:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:06:36.139088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2ea3-be7a-4321-b4d4-902aaccf3d65 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:36.139541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2ea3-be7a-4321-b4d4-902aaccf3d65 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:36.139912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2ea3-be7a-4321-b4d4-902aaccf3d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:36.191613 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-487d2ea3-be7a-4321-b4d4-902aaccf3d65 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:36.219552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-487d2ea3-be7a-4321-b4d4-902aaccf3d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:36.222132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 770/1543] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:36.248014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd62ef-3fb0-4b25-962c-91eea4e3317f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:36.249522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6ccd62ef-3fb0-4b25-962c-91eea4e3317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:36.372642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 774/1544] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:06:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:36.373784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d2ea3-be7a-4321-b4d4-902aaccf3d65 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:36.375415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-487d2ea3-be7a-4321-b4d4-902aaccf3d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:36.379565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd62ef-3fb0-4b25-962c-91eea4e3317f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:36.390876 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d2ea3-be7a-4321-b4d4-902aaccf3d65 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:36.395115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3769b1-2e24-4fe7-b965-fefa627819e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:36.395757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3769b1-2e24-4fe7-b965-fefa627819e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:36.396039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3769b1-2e24-4fe7-b965-fefa627819e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:36.396312 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3769b1-2e24-4fe7-b965-fefa627819e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:36.398021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:36.398882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:36.399369 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:36.399996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] PATH_INFO: `/v3/projects/adb7f97c4bfa4323881e08de6d83bc7d` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:36.401425 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=adb7f97c4bfa4323881e08de6d83bc7d)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:36.404112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:36.431435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3769b1-2e24-4fe7-b965-fefa627819e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:36.432927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3769b1-2e24-4fe7-b965-fefa627819e7 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:36.477175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 771/1545] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:06:36.478543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3769b1-2e24-4fe7-b965-fefa627819e7 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:36.482076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7b3769b1-2e24-4fe7-b965-fefa627819e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:36.518965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adb7f97c4bfa4323881e08de6d83bc7d', 'request_id': 'req-63a61a33-4cf0-4034-93a9-7689fc1d43bf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.520810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3769b1-2e24-4fe7-b965-fefa627819e7 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:36.535652 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adb7f97c4bfa4323881e08de6d83bc7d', 'request_id': 'req-63a61a33-4cf0-4034-93a9-7689fc1d43bf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.547867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adb7f97c4bfa4323881e08de6d83bc7d', 'request_id': 'req-63a61a33-4cf0-4034-93a9-7689fc1d43bf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.576462 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] The token cache is being invalidate because project adb7f97c4bfa4323881e08de6d83bc7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:36.576860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.579236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.579377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:36.582787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 775/1546] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:36 2022] DELETE /identity/v3/projects/adb7f97c4bfa4323881e08de6d83bc7d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:36.585878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:36.594196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:36.629341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:36.642564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:36.643614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:36.643903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:36.644933 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] PATH_INFO: `/v3/users/6a3a54b717664e0ea3308fb57ae0272f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:36.654903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3a54b717664e0ea3308fb57ae0272f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:36.657651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:36.768594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a61a33-4cf0-4034-93a9-7689fc1d43bf admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:36.770787 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3a54b717664e0ea3308fb57ae0272f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.771886 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3a54b717664e0ea3308fb57ae0272f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.782932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:36.783631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:36.783909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:36.784182 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:36.788264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:36.796838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3a54b717664e0ea3308fb57ae0272f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.806515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:36.808318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:36.809883 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3a54b717664e0ea3308fb57ae0272f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.821020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 776/1547] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:36.822938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-330caeb9-de9b-442d-b14c-a9ca6c62e530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3a54b717664e0ea3308fb57ae0272f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:36.833501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:36.837183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 772/1548] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:36 2022] DELETE /identity/v3/users/6a3a54b717664e0ea3308fb57ae0272f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:36.838292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:36.839387 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbf1e7-a28c-460b-abce-f81cd61b9ed3 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:36.839692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbf1e7-a28c-460b-abce-f81cd61b9ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:36.840087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbf1e7-a28c-460b-abce-f81cd61b9ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:36.868221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb21bf5-5142-4dbc-a8a1-3e42bd7a97cc service nova] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:36.878101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a2cbf1e7-a28c-460b-abce-f81cd61b9ed3 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:36.888371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a2cbf1e7-a28c-460b-abce-f81cd61b9ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:36.991092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:36.991757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:36.992105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:36.992381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] PATH_INFO: `/v3/users/abed80d0ed3f450a94bb7bd993de7c90` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:36.993795 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abed80d0ed3f450a94bb7bd993de7c90)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:37.005245 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:37.036954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 773/1549] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:37.065109 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbf1e7-a28c-460b-abce-f81cd61b9ed3 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:37.072535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a2cbf1e7-a28c-460b-abce-f81cd61b9ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:37.134091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abed80d0ed3f450a94bb7bd993de7c90'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.136459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abed80d0ed3f450a94bb7bd993de7c90'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.159806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abed80d0ed3f450a94bb7bd993de7c90'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.170458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abed80d0ed3f450a94bb7bd993de7c90'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.171502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abed80d0ed3f450a94bb7bd993de7c90'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.195908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 777/1550] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:36 2022] DELETE /identity/v3/users/abed80d0ed3f450a94bb7bd993de7c90 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:37.217831 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:37.223226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbf1e7-a28c-460b-abce-f81cd61b9ed3 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:37.347210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6b07e-b23a-4c0e-86fa-1a571b56752d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:37.348309 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6b07e-b23a-4c0e-86fa-1a571b56752d service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:37.348710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6b07e-b23a-4c0e-86fa-1a571b56752d service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:37.349081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6b07e-b23a-4c0e-86fa-1a571b56752d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:37.354895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-ebc6b07e-b23a-4c0e-86fa-1a571b56752d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:37.543928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6b07e-b23a-4c0e-86fa-1a571b56752d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:37.545514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6b07e-b23a-4c0e-86fa-1a571b56752d service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:37.567429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 774/1551] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:37.602424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:37.605759 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:37.627128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b863559-2e65-4bdf-9a84-d88d43e84fe3 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:37.635052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:37.636019 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:37.636122 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:37.636333 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] PATH_INFO: `/v3/projects/a4221290e7714968889e8772bfa77a7c` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:37.638803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4221290e7714968889e8772bfa77a7c)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:37.641867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:37.676523 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:37.719890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4221290e7714968889e8772bfa77a7c', 'request_id': 'req-6b555ecd-8741-4dc0-951f-3024aee0afe0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.733157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4221290e7714968889e8772bfa77a7c', 'request_id': 'req-6b555ecd-8741-4dc0-951f-3024aee0afe0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.742362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4221290e7714968889e8772bfa77a7c', 'request_id': 'req-6b555ecd-8741-4dc0-951f-3024aee0afe0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.775394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] The token cache is being invalidate because project a4221290e7714968889e8772bfa77a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:37.775957 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.777515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.780458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 778/1552] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:37 2022] DELETE /identity/v3/projects/a4221290e7714968889e8772bfa77a7c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:37.784205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6b07e-b23a-4c0e-86fa-1a571b56752d service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:37.795762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6b07e-b23a-4c0e-86fa-1a571b56752d service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:37.853963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:37.854557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:37.854840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:37.855486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] PATH_INFO: `/v3/users/07ac544d18974577b47f98ca25822abf` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:37.856292 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ac544d18974577b47f98ca25822abf)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:37.858241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:37.923627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ac544d18974577b47f98ca25822abf'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.926689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ac544d18974577b47f98ca25822abf'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.936668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:37.945527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ac544d18974577b47f98ca25822abf'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.956734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ac544d18974577b47f98ca25822abf'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.964853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ac544d18974577b47f98ca25822abf'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:37.975763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 775/1553] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:37 2022] DELETE /identity/v3/users/07ac544d18974577b47f98ca25822abf => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:38.033932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.036310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:38.051416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b555ecd-8741-4dc0-951f-3024aee0afe0 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.156739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.157843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.158334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.159696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] PATH_INFO: `/v3/projects/0ad0bc549fb74104ae7d79db2388017e` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.161014 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad0bc549fb74104ae7d79db2388017e)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.167467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.197129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.198931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:38.221904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634cf19-2843-4762-b9e1-21a5c5baea9a admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.231649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.232061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.232270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.232548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] PATH_INFO: `/v3/projects/a2ec8c0835b84fc3bf0da9af314da8eb` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.234724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ec8c0835b84fc3bf0da9af314da8eb)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.239027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.271956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad0bc549fb74104ae7d79db2388017e', 'request_id': 'req-74e38fa0-32a2-49d1-91a5-03d407a62bb5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.280258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad0bc549fb74104ae7d79db2388017e', 'request_id': 'req-74e38fa0-32a2-49d1-91a5-03d407a62bb5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.288046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad0bc549fb74104ae7d79db2388017e', 'request_id': 'req-74e38fa0-32a2-49d1-91a5-03d407a62bb5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.300613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ec8c0835b84fc3bf0da9af314da8eb', 'request_id': 'req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.301747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] The token cache is being invalidate because project 0ad0bc549fb74104ae7d79db2388017e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:38.302039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.302785 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-74e38fa0-32a2-49d1-91a5-03d407a62bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.308143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 779/1554] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:38 2022] DELETE /identity/v3/projects/0ad0bc549fb74104ae7d79db2388017e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:38.313063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ec8c0835b84fc3bf0da9af314da8eb', 'request_id': 'req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.313207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b1513-66c6-4e46-8ba5-79054b60a590 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.313519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b1513-66c6-4e46-8ba5-79054b60a590 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.313788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b1513-66c6-4e46-8ba5-79054b60a590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.322947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ec8c0835b84fc3bf0da9af314da8eb', 'request_id': 'req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.336656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8c2b1513-66c6-4e46-8ba5-79054b60a590 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:38.337928 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8c2b1513-66c6-4e46-8ba5-79054b60a590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:38.348331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] The token cache is being invalidate because project a2ec8c0835b84fc3bf0da9af314da8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:38.349682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.351036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.352785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 776/1555] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:38 2022] DELETE /identity/v3/projects/a2ec8c0835b84fc3bf0da9af314da8eb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:38.353466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.354769 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:38.377570 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:38.412882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 780/1556] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:38.412882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b1513-66c6-4e46-8ba5-79054b60a590 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.412882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8c2b1513-66c6-4e46-8ba5-79054b60a590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:38.421257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b1513-66c6-4e46-8ba5-79054b60a590 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.421943 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320bba7-9ce3-431e-96a8-0a2dc43c6cfe admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.426522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.426974 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.427186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.427589 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.427678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.427864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] PATH_INFO: `/v3/users/7efd22c913ac4e8381ad46981c8956f6` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.427955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.428130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] PATH_INFO: `/v3/projects/4b0943e586384e1ba7c63555d70686ee` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.428978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7efd22c913ac4e8381ad46981c8956f6)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.429221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0943e586384e1ba7c63555d70686ee)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.430873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.431138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.483806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7efd22c913ac4e8381ad46981c8956f6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.484556 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0943e586384e1ba7c63555d70686ee', 'request_id': 'req-4aac2c59-02c3-4bb8-8897-8da247dba4a9'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.484761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7efd22c913ac4e8381ad46981c8956f6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.491716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0943e586384e1ba7c63555d70686ee', 'request_id': 'req-4aac2c59-02c3-4bb8-8897-8da247dba4a9'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.495527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:38.498612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7efd22c913ac4e8381ad46981c8956f6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.502457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0943e586384e1ba7c63555d70686ee', 'request_id': 'req-4aac2c59-02c3-4bb8-8897-8da247dba4a9'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.508459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7efd22c913ac4e8381ad46981c8956f6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.509512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-103421f8-216f-4fcb-b498-2da168eca64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7efd22c913ac4e8381ad46981c8956f6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.522591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] The token cache is being invalidate because project 4b0943e586384e1ba7c63555d70686ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:38.522930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.523717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4aac2c59-02c3-4bb8-8897-8da247dba4a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:38.526709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 777/1557] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:38 2022] DELETE /identity/v3/projects/4b0943e586384e1ba7c63555d70686ee => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:38.527906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0699e0-69f5-4fc6-8a00-689d62faf518 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.528191 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0699e0-69f5-4fc6-8a00-689d62faf518 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.528462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0699e0-69f5-4fc6-8a00-689d62faf518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.529018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 781/1558] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:38 2022] DELETE /identity/v3/users/7efd22c913ac4e8381ad46981c8956f6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:38.535165 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987bae2-8f7b-4224-9273-d4af8db09b7f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.535456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987bae2-8f7b-4224-9273-d4af8db09b7f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.535864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987bae2-8f7b-4224-9273-d4af8db09b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.557234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-aa0699e0-69f5-4fc6-8a00-689d62faf518 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:38.558723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-aa0699e0-69f5-4fc6-8a00-689d62faf518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:38.570215 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2987bae2-8f7b-4224-9273-d4af8db09b7f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:38.571531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2987bae2-8f7b-4224-9273-d4af8db09b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:38.649046 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 778/1559] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:38.653791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0699e0-69f5-4fc6-8a00-689d62faf518 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.674627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 782/1560] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:38.682560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0699e0-69f5-4fc6-8a00-689d62faf518 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.682780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8510a-0ea5-422d-893e-ad62417e77c6 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.683073 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8510a-0ea5-422d-893e-ad62417e77c6 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.683337 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8510a-0ea5-422d-893e-ad62417e77c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.690865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bca61-f326-4a66-8294-725fa87dffa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.692053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bca61-f326-4a66-8294-725fa87dffa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.692527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bca61-f326-4a66-8294-725fa87dffa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.693103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bca61-f326-4a66-8294-725fa87dffa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.694484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bca61-f326-4a66-8294-725fa87dffa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:38.694805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bca61-f326-4a66-8294-725fa87dffa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.696981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bca61-f326-4a66-8294-725fa87dffa2 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.709200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6ff8510a-0ea5-422d-893e-ad62417e77c6 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:38.710431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6ff8510a-0ea5-422d-893e-ad62417e77c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:38.714164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 779/1561] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:38.715232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bca61-f326-4a66-8294-725fa87dffa2 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.723018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 783/1562] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:38.724314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8510a-0ea5-422d-893e-ad62417e77c6 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.732665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bca61-f326-4a66-8294-725fa87dffa2 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.733263 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addb419-6b41-4ff4-b8ed-64928ddcc547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.733837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addb419-6b41-4ff4-b8ed-64928ddcc547 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.734171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addb419-6b41-4ff4-b8ed-64928ddcc547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.734447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addb419-6b41-4ff4-b8ed-64928ddcc547 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.735163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addb419-6b41-4ff4-b8ed-64928ddcc547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.735441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8510a-0ea5-422d-893e-ad62417e77c6 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.737933 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addb419-6b41-4ff4-b8ed-64928ddcc547 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.741670 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef03f2-bab6-4e54-abb7-7a416e6919ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.743159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef03f2-bab6-4e54-abb7-7a416e6919ff admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.743956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef03f2-bab6-4e54-abb7-7a416e6919ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.744410 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef03f2-bab6-4e54-abb7-7a416e6919ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.745249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef03f2-bab6-4e54-abb7-7a416e6919ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:38.745863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef03f2-bab6-4e54-abb7-7a416e6919ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.747765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef03f2-bab6-4e54-abb7-7a416e6919ff admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.777669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 784/1563] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:06:38.778282 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 780/1564] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:38 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:38.778282 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addb419-6b41-4ff4-b8ed-64928ddcc547 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.780958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef03f2-bab6-4e54-abb7-7a416e6919ff admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.815731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef03f2-bab6-4e54-abb7-7a416e6919ff admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.855981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addb419-6b41-4ff4-b8ed-64928ddcc547 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.861296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b60c-775a-4d3b-96a3-c4ed2dcf2672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.862305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b60c-775a-4d3b-96a3-c4ed2dcf2672 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.862591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b60c-775a-4d3b-96a3-c4ed2dcf2672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.862890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169b60c-775a-4d3b-96a3-c4ed2dcf2672 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.863569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169b60c-775a-4d3b-96a3-c4ed2dcf2672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.865906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169b60c-775a-4d3b-96a3-c4ed2dcf2672 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.868234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:38.891975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 781/1565] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:38 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:38.896702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b60c-775a-4d3b-96a3-c4ed2dcf2672 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.904766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078540c8-4014-4a4f-877a-843aae34c23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.905561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078540c8-4014-4a4f-877a-843aae34c23d admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.905917 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078540c8-4014-4a4f-877a-843aae34c23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.906022 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078540c8-4014-4a4f-877a-843aae34c23d admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.906618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078540c8-4014-4a4f-877a-843aae34c23d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.908811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169b60c-775a-4d3b-96a3-c4ed2dcf2672 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.909768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078540c8-4014-4a4f-877a-843aae34c23d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.914117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aff820-77bd-4d6c-9d82-011481041f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:38.914765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aff820-77bd-4d6c-9d82-011481041f45 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:38.915073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aff820-77bd-4d6c-9d82-011481041f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:38.915335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aff820-77bd-4d6c-9d82-011481041f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:38.916371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aff820-77bd-4d6c-9d82-011481041f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:38.917552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aff820-77bd-4d6c-9d82-011481041f45 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:38.928491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 782/1566] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:06:38.933011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aff820-77bd-4d6c-9d82-011481041f45 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.945411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aff820-77bd-4d6c-9d82-011481041f45 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:38.950446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 785/1567] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:38 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:38.951483 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078540c8-4014-4a4f-877a-843aae34c23d admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:38.984275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:39.024282 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555294d-9213-4bbf-98ef-1802466ac990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.024928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555294d-9213-4bbf-98ef-1802466ac990 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.025298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555294d-9213-4bbf-98ef-1802466ac990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.025651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555294d-9213-4bbf-98ef-1802466ac990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.026310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555294d-9213-4bbf-98ef-1802466ac990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.026832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078540c8-4014-4a4f-877a-843aae34c23d admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.028039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555294d-9213-4bbf-98ef-1802466ac990 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.033817 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7709186-ac59-4df7-b496-a0f29f198af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.034918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7709186-ac59-4df7-b496-a0f29f198af5 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.035525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7709186-ac59-4df7-b496-a0f29f198af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.035804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7709186-ac59-4df7-b496-a0f29f198af5 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.039281 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7709186-ac59-4df7-b496-a0f29f198af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.041312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7709186-ac59-4df7-b496-a0f29f198af5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.051256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 783/1568] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:39.051256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555294d-9213-4bbf-98ef-1802466ac990 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.053690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4555294d-9213-4bbf-98ef-1802466ac990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:39.068764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 786/1569] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:38 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:39.068940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555294d-9213-4bbf-98ef-1802466ac990 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.071918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7709186-ac59-4df7-b496-a0f29f198af5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.073817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.074464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.074741 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.075043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] PATH_INFO: `/v3/projects/8481d4b15a834f67bbe73e991293df0a` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.076168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8481d4b15a834f67bbe73e991293df0a)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.078082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.084103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7709186-ac59-4df7-b496-a0f29f198af5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.088941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b938d6b-35b1-47d1-bc53-37f42cdbcfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.089697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b938d6b-35b1-47d1-bc53-37f42cdbcfe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.090039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b938d6b-35b1-47d1-bc53-37f42cdbcfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.090393 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b938d6b-35b1-47d1-bc53-37f42cdbcfe4 admin admin] PATH_INFO: `/v3/projects/5d0a984b904a4641a4b300289b40aa87/users/24de3954923640099f06fb50b7829d18/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.106637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b938d6b-35b1-47d1-bc53-37f42cdbcfe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d0a984b904a4641a4b300289b40aa87, user_id=24de3954923640099f06fb50b7829d18, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.109470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b938d6b-35b1-47d1-bc53-37f42cdbcfe4 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.123068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 787/1570] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:39 2022] PUT /identity/v3/projects/5d0a984b904a4641a4b300289b40aa87/users/24de3954923640099f06fb50b7829d18/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:39.123860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b938d6b-35b1-47d1-bc53-37f42cdbcfe4 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.134934 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8481d4b15a834f67bbe73e991293df0a', 'request_id': 'req-cd2e29db-e594-47c5-8475-5a2d3745e638'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:39.141494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8481d4b15a834f67bbe73e991293df0a', 'request_id': 'req-cd2e29db-e594-47c5-8475-5a2d3745e638'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:39.149515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8481d4b15a834f67bbe73e991293df0a', 'request_id': 'req-cd2e29db-e594-47c5-8475-5a2d3745e638'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:39.161636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] The token cache is being invalidate because project 8481d4b15a834f67bbe73e991293df0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:39.161928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:39.162581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cd2e29db-e594-47c5-8475-5a2d3745e638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:39.164162 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 784/1571] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:39 2022] DELETE /identity/v3/projects/8481d4b15a834f67bbe73e991293df0a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:39.167113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070405f-c305-42bc-b6d9-c5bdd8180bd8 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.167113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070405f-c305-42bc-b6d9-c5bdd8180bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.167113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070405f-c305-42bc-b6d9-c5bdd8180bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.189279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b938d6b-35b1-47d1-bc53-37f42cdbcfe4 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.193671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d070405f-c305-42bc-b6d9-c5bdd8180bd8 None None] MFA Rules not processed for user `24de3954923640099f06fb50b7829d18`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:39.193888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384bab7-5be9-4213-b76a-4572471e1fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.194493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384bab7-5be9-4213-b76a-4572471e1fda admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.194775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384bab7-5be9-4213-b76a-4572471e1fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.194872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d070405f-c305-42bc-b6d9-c5bdd8180bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:39.195076 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384bab7-5be9-4213-b76a-4572471e1fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.195718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384bab7-5be9-4213-b76a-4572471e1fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.197332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384bab7-5be9-4213-b76a-4572471e1fda admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.207964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 788/1572] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:39.209132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d74c1-9b57-4235-bf2e-fab2eab57fa3 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.209408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d74c1-9b57-4235-bf2e-fab2eab57fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.209676 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d74c1-9b57-4235-bf2e-fab2eab57fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.229874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-578d74c1-9b57-4235-bf2e-fab2eab57fa3 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:39.231352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-578d74c1-9b57-4235-bf2e-fab2eab57fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:39.239152 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 789/1573] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:39.239912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d74c1-9b57-4235-bf2e-fab2eab57fa3 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.241128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-578d74c1-9b57-4235-bf2e-fab2eab57fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:39.252874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d74c1-9b57-4235-bf2e-fab2eab57fa3 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.257466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007e8cb-fcb2-47f2-82ef-1862165b20fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.258044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007e8cb-fcb2-47f2-82ef-1862165b20fe admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.258319 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007e8cb-fcb2-47f2-82ef-1862165b20fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.258718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007e8cb-fcb2-47f2-82ef-1862165b20fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.259253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007e8cb-fcb2-47f2-82ef-1862165b20fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.260825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007e8cb-fcb2-47f2-82ef-1862165b20fe admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.267385 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:39.269543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 785/1574] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:39.270768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a62b10-d8e6-4e62-a90d-3c25bbf310ac None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.271154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a62b10-d8e6-4e62-a90d-3c25bbf310ac None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.271352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a62b10-d8e6-4e62-a90d-3c25bbf310ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.276672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 790/1575] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:39.277679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007e8cb-fcb2-47f2-82ef-1862165b20fe admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.279717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9007e8cb-fcb2-47f2-82ef-1862165b20fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:39.293776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007e8cb-fcb2-47f2-82ef-1862165b20fe admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.295102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-28a62b10-d8e6-4e62-a90d-3c25bbf310ac None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:39.296305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-28a62b10-d8e6-4e62-a90d-3c25bbf310ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:39.300221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c23ee2-d3e0-4f8c-9976-9446be19da3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.300788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c23ee2-d3e0-4f8c-9976-9446be19da3a admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.301067 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c23ee2-d3e0-4f8c-9976-9446be19da3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.301341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c23ee2-d3e0-4f8c-9976-9446be19da3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.302040 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c23ee2-d3e0-4f8c-9976-9446be19da3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.304050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c23ee2-d3e0-4f8c-9976-9446be19da3a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.304506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 786/1576] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:39.305340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a62b10-d8e6-4e62-a90d-3c25bbf310ac None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.315783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a62b10-d8e6-4e62-a90d-3c25bbf310ac None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.320417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c3468-f716-4536-b239-8aca9478ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.321075 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c3468-f716-4536-b239-8aca9478ec86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.321354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c3468-f716-4536-b239-8aca9478ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.321627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c3468-f716-4536-b239-8aca9478ec86 admin admin] PATH_INFO: `/v3/projects/098be4758aeb4378afdd49c3e99c59a7/users/9bf29df38e1f4d3ab23d93321e8f491d/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.336805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 791/1577] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:39 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:39.337646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c23ee2-d3e0-4f8c-9976-9446be19da3a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.338904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c3468-f716-4536-b239-8aca9478ec86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=098be4758aeb4378afdd49c3e99c59a7, user_id=9bf29df38e1f4d3ab23d93321e8f491d, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.341705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c3468-f716-4536-b239-8aca9478ec86 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.356313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 787/1578] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:39 2022] PUT /identity/v3/projects/098be4758aeb4378afdd49c3e99c59a7/users/9bf29df38e1f4d3ab23d93321e8f491d/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:39.360393 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c3468-f716-4536-b239-8aca9478ec86 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.371442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:39.412860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c23ee2-d3e0-4f8c-9976-9446be19da3a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.479673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d3953-10b2-4c79-b62c-932d5090b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.480305 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d3953-10b2-4c79-b62c-932d5090b39d admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.480580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d3953-10b2-4c79-b62c-932d5090b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.480996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d3953-10b2-4c79-b62c-932d5090b39d admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.481660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d3953-10b2-4c79-b62c-932d5090b39d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:39.482102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d3953-10b2-4c79-b62c-932d5090b39d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.483919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d3953-10b2-4c79-b62c-932d5090b39d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.498634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 792/1579] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:39.498634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54987965-57ae-4607-ab38-5ac8ba0e9c6b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.498634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54987965-57ae-4607-ab38-5ac8ba0e9c6b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.499016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54987965-57ae-4607-ab38-5ac8ba0e9c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.519922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-54987965-57ae-4607-ab38-5ac8ba0e9c6b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:39.521378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-54987965-57ae-4607-ab38-5ac8ba0e9c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:39.528955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 793/1580] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:39.529692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54987965-57ae-4607-ab38-5ac8ba0e9c6b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.539524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54987965-57ae-4607-ab38-5ac8ba0e9c6b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.544513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f767201-0556-4583-8b31-79aac842b5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.545097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f767201-0556-4583-8b31-79aac842b5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.545369 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f767201-0556-4583-8b31-79aac842b5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.545632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f767201-0556-4583-8b31-79aac842b5d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.546245 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f767201-0556-4583-8b31-79aac842b5d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.548157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f767201-0556-4583-8b31-79aac842b5d5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.555095 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c3468-f716-4536-b239-8aca9478ec86 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.560009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541865cf-0220-4cb5-a10e-56920f78f138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.560609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541865cf-0220-4cb5-a10e-56920f78f138 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.560880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541865cf-0220-4cb5-a10e-56920f78f138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.561150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541865cf-0220-4cb5-a10e-56920f78f138 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.562065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541865cf-0220-4cb5-a10e-56920f78f138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.564018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541865cf-0220-4cb5-a10e-56920f78f138 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.576534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 794/1581] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:39 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:39.577614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f767201-0556-4583-8b31-79aac842b5d5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.579240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7f767201-0556-4583-8b31-79aac842b5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:39.592344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 788/1582] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:39 2022] POST /identity/v3/users => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:39.592344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541865cf-0220-4cb5-a10e-56920f78f138 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.600258 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541865cf-0220-4cb5-a10e-56920f78f138 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.639591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f767201-0556-4583-8b31-79aac842b5d5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.643806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e969935-d69a-43b4-9fac-1aa9eb8d5fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.644438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e969935-d69a-43b4-9fac-1aa9eb8d5fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.644904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e969935-d69a-43b4-9fac-1aa9eb8d5fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.645004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e969935-d69a-43b4-9fac-1aa9eb8d5fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.649100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:39.656099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df0bdd-a3a0-40e2-9b57-cb4dd76f0af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.656685 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df0bdd-a3a0-40e2-9b57-cb4dd76f0af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.656962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df0bdd-a3a0-40e2-9b57-cb4dd76f0af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.657224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df0bdd-a3a0-40e2-9b57-cb4dd76f0af4 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.657841 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df0bdd-a3a0-40e2-9b57-cb4dd76f0af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.660432 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df0bdd-a3a0-40e2-9b57-cb4dd76f0af4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.681564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 789/1583] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:39 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:39.686479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df0bdd-a3a0-40e2-9b57-cb4dd76f0af4 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.698148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df0bdd-a3a0-40e2-9b57-cb4dd76f0af4 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.703805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44053f37-74dd-43a5-bd37-1804f82bce6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.704411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44053f37-74dd-43a5-bd37-1804f82bce6f admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.704745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44053f37-74dd-43a5-bd37-1804f82bce6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.705557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44053f37-74dd-43a5-bd37-1804f82bce6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.706203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44053f37-74dd-43a5-bd37-1804f82bce6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.708279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44053f37-74dd-43a5-bd37-1804f82bce6f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.717463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e969935-d69a-43b4-9fac-1aa9eb8d5fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.718900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e969935-d69a-43b4-9fac-1aa9eb8d5fc1 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.719583 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 790/1584] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:39.720220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44053f37-74dd-43a5-bd37-1804f82bce6f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.730411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44053f37-74dd-43a5-bd37-1804f82bce6f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.735345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d0b80-f7cb-40f9-8354-6edc6918fc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.735947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d0b80-f7cb-40f9-8354-6edc6918fc8a admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.737012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d0b80-f7cb-40f9-8354-6edc6918fc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.737285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d0b80-f7cb-40f9-8354-6edc6918fc8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.737939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d0b80-f7cb-40f9-8354-6edc6918fc8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.739536 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d0b80-f7cb-40f9-8354-6edc6918fc8a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.740390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 795/1585] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:39.741046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e969935-d69a-43b4-9fac-1aa9eb8d5fc1 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.750823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e969935-d69a-43b4-9fac-1aa9eb8d5fc1 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.754967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 791/1586] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:39.755907 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d0b80-f7cb-40f9-8354-6edc6918fc8a admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.757867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4639e6-5223-42b2-b5e9-c48e5355e808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.758542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4639e6-5223-42b2-b5e9-c48e5355e808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.758816 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4639e6-5223-42b2-b5e9-c48e5355e808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.759498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4639e6-5223-42b2-b5e9-c48e5355e808 admin admin] PATH_INFO: `/v3/projects/250e97b9047a41e6b6530175ef5f9e28/users/b012356b5703446e83ec1e44520aae24/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.770329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d0b80-f7cb-40f9-8354-6edc6918fc8a admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.778133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4639e6-5223-42b2-b5e9-c48e5355e808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250e97b9047a41e6b6530175ef5f9e28, user_id=b012356b5703446e83ec1e44520aae24, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.779480 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c3776-6e51-4998-9fbc-a549bf4017aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.780377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c3776-6e51-4998-9fbc-a549bf4017aa admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.780844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c3776-6e51-4998-9fbc-a549bf4017aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.781016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4639e6-5223-42b2-b5e9-c48e5355e808 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.781520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c3776-6e51-4998-9fbc-a549bf4017aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.782505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c3776-6e51-4998-9fbc-a549bf4017aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.784983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c3776-6e51-4998-9fbc-a549bf4017aa admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.798091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 792/1587] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:39.804024 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c3776-6e51-4998-9fbc-a549bf4017aa admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.807128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 796/1588] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:39 2022] PUT /identity/v3/projects/250e97b9047a41e6b6530175ef5f9e28/users/b012356b5703446e83ec1e44520aae24/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:39.812393 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4639e6-5223-42b2-b5e9-c48e5355e808 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:39.821446 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c3776-6e51-4998-9fbc-a549bf4017aa admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.911544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:39.920864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61279049-b091-49a3-93f5-dd026d6e2bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.921538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61279049-b091-49a3-93f5-dd026d6e2bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.922101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61279049-b091-49a3-93f5-dd026d6e2bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.922434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61279049-b091-49a3-93f5-dd026d6e2bc2 admin admin] PATH_INFO: `/v3/projects/86d5fe15d7a64d17bec96264a8eba963/users/4f033761116c4a36bf77f7eec4c469c6/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.940230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61279049-b091-49a3-93f5-dd026d6e2bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d5fe15d7a64d17bec96264a8eba963, user_id=4f033761116c4a36bf77f7eec4c469c6, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.940687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4639e6-5223-42b2-b5e9-c48e5355e808 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:39.943697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61279049-b091-49a3-93f5-dd026d6e2bc2 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.948618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea6325-13e2-4aa8-a507-9a884e7d9e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:39.961952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea6325-13e2-4aa8-a507-9a884e7d9e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.962341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea6325-13e2-4aa8-a507-9a884e7d9e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.962624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea6325-13e2-4aa8-a507-9a884e7d9e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.964982 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea6325-13e2-4aa8-a507-9a884e7d9e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:39.968329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea6325-13e2-4aa8-a507-9a884e7d9e6e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:39.970063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 793/1589] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:39 2022] PUT /identity/v3/projects/86d5fe15d7a64d17bec96264a8eba963/users/4f033761116c4a36bf77f7eec4c469c6/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:39.983873 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c127c-0b53-4d59-bf84-84eae702bb57 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:39.984608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c127c-0b53-4d59-bf84-84eae702bb57 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:39.984896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c127c-0b53-4d59-bf84-84eae702bb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:39.994739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 797/1590] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:39.995934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea6325-13e2-4aa8-a507-9a884e7d9e6e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:40.011584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea6325-13e2-4aa8-a507-9a884e7d9e6e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:40.017044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-947c127c-0b53-4d59-bf84-84eae702bb57 None None] MFA Rules not processed for user `4f033761116c4a36bf77f7eec4c469c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:40.018918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-947c127c-0b53-4d59-bf84-84eae702bb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:40.114734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d37ce-b826-41db-9e3e-dfb52a64ac5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:40.115861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d37ce-b826-41db-9e3e-dfb52a64ac5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:40.116223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d37ce-b826-41db-9e3e-dfb52a64ac5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:40.116799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d37ce-b826-41db-9e3e-dfb52a64ac5f admin admin] PATH_INFO: `/v3/projects/250e97b9047a41e6b6530175ef5f9e28/users/b012356b5703446e83ec1e44520aae24/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:40.122970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d37ce-b826-41db-9e3e-dfb52a64ac5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=250e97b9047a41e6b6530175ef5f9e28, user_id=b012356b5703446e83ec1e44520aae24, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:40.127445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 794/1591] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:40.135217 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d37ce-b826-41db-9e3e-dfb52a64ac5f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:40.141865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c127c-0b53-4d59-bf84-84eae702bb57 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:40.144249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-947c127c-0b53-4d59-bf84-84eae702bb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:40.156477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 798/1592] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:39 2022] PUT /identity/v3/projects/250e97b9047a41e6b6530175ef5f9e28/users/b012356b5703446e83ec1e44520aae24/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:40.161718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d904c5-ac2e-44d0-ad3d-8b90c6bcec4f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:40.162207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d904c5-ac2e-44d0-ad3d-8b90c6bcec4f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:40.162624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d904c5-ac2e-44d0-ad3d-8b90c6bcec4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:40.185330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:40.219202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e1d904c5-ac2e-44d0-ad3d-8b90c6bcec4f None None] MFA Rules not processed for user `b012356b5703446e83ec1e44520aae24`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:40.222640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e1d904c5-ac2e-44d0-ad3d-8b90c6bcec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:40.226717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c127c-0b53-4d59-bf84-84eae702bb57 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:40.285890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21466e-b206-4083-8f28-602ee50c633c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:40.286829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21466e-b206-4083-8f28-602ee50c633c service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:40.287133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21466e-b206-4083-8f28-602ee50c633c service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:40.287405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21466e-b206-4083-8f28-602ee50c633c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:40.310462 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 799/1593] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:40.325608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d904c5-ac2e-44d0-ad3d-8b90c6bcec4f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:40.327887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e1d904c5-ac2e-44d0-ad3d-8b90c6bcec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:40.331541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:40.350907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d904c5-ac2e-44d0-ad3d-8b90c6bcec4f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:40.353195 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5153d-cb19-4aaa-97dc-2302fe657025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:40.353817 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5153d-cb19-4aaa-97dc-2302fe657025 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:40.354094 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5153d-cb19-4aaa-97dc-2302fe657025 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:40.354361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5153d-cb19-4aaa-97dc-2302fe657025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:40.358363 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21466e-b206-4083-8f28-602ee50c633c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:40.360011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21466e-b206-4083-8f28-602ee50c633c service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:40.365828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5153d-cb19-4aaa-97dc-2302fe657025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:40.367822 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5153d-cb19-4aaa-97dc-2302fe657025 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:40.370886 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 795/1594] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:40.381436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 800/1595] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:40 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:40.553204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21466e-b206-4083-8f28-602ee50c633c service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:40.639607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21466e-b206-4083-8f28-602ee50c633c service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:40.650319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:40.652904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4650ee-6a98-4439-9c2d-0f38286a49fb None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:40.653227 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4650ee-6a98-4439-9c2d-0f38286a49fb None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:40.653820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4650ee-6a98-4439-9c2d-0f38286a49fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:40.655327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:40.656678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:40.658836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] PATH_INFO: `/v3/users/24de3954923640099f06fb50b7829d18` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:40.662384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=24de3954923640099f06fb50b7829d18)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:40.664728 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:40.720171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-be4650ee-6a98-4439-9c2d-0f38286a49fb None None] MFA Rules not processed for user `9bf29df38e1f4d3ab23d93321e8f491d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:40.725544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-be4650ee-6a98-4439-9c2d-0f38286a49fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:40.796619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24de3954923640099f06fb50b7829d18'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:40.797559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24de3954923640099f06fb50b7829d18'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:40.833331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24de3954923640099f06fb50b7829d18'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:40.842841 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24de3954923640099f06fb50b7829d18'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:40.856442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24de3954923640099f06fb50b7829d18'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:40.867269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 796/1596] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:40 2022] DELETE /identity/v3/users/24de3954923640099f06fb50b7829d18 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:40.868274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:40.869551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 801/1597] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:40.871420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:40.874477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49845ceb-b06d-4d2f-a7c2-29267f4cc234 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:40.874758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49845ceb-b06d-4d2f-a7c2-29267f4cc234 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:40.875049 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49845ceb-b06d-4d2f-a7c2-29267f4cc234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:40.896732 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-49845ceb-b06d-4d2f-a7c2-29267f4cc234 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:40.897947 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-49845ceb-b06d-4d2f-a7c2-29267f4cc234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:40.937450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eeebb4-b5ec-4f98-bfc6-4edf53e91bcd admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:40.941288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f5e9-8840-4dfa-b262-4bfb532af558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:40.941903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f5e9-8840-4dfa-b262-4bfb532af558 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:40.942187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f5e9-8840-4dfa-b262-4bfb532af558 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:40.942460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f5e9-8840-4dfa-b262-4bfb532af558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:40.944208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-db50f5e9-8840-4dfa-b262-4bfb532af558 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:40.955894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:40.968111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 802/1598] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:40.968111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49845ceb-b06d-4d2f-a7c2-29267f4cc234 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:40.975750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-49845ceb-b06d-4d2f-a7c2-29267f4cc234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:40.988511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49845ceb-b06d-4d2f-a7c2-29267f4cc234 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:40.994018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3cad1-a722-4f62-ab71-bece848694a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:40.994640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3cad1-a722-4f62-ab71-bece848694a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:40.994916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3cad1-a722-4f62-ab71-bece848694a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:40.995236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3cad1-a722-4f62-ab71-bece848694a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:40.995873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3cad1-a722-4f62-ab71-bece848694a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:40.997948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3cad1-a722-4f62-ab71-bece848694a1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:41.003785 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:41.024870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50f5e9-8840-4dfa-b262-4bfb532af558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:41.026314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50f5e9-8840-4dfa-b262-4bfb532af558 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:41.030268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 803/1599] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:40 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:41.032531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3cad1-a722-4f62-ab71-bece848694a1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:41.035480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-99c3cad1-a722-4f62-ab71-bece848694a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:41.037891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 797/1600] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:41.038757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f5e9-8840-4dfa-b262-4bfb532af558 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:41.049687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f5e9-8840-4dfa-b262-4bfb532af558 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:41.135003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552be6d-11e9-4e65-b021-895d7ca807d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:41.135949 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552be6d-11e9-4e65-b021-895d7ca807d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:41.136294 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552be6d-11e9-4e65-b021-895d7ca807d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:41.136637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552be6d-11e9-4e65-b021-895d7ca807d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:41.139631 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552be6d-11e9-4e65-b021-895d7ca807d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:41.141822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552be6d-11e9-4e65-b021-895d7ca807d4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:41.154744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3cad1-a722-4f62-ab71-bece848694a1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:41.160376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b6b83-b401-4938-92ee-bebfe4fa477a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:41.161415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6b83-b401-4938-92ee-bebfe4fa477a service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:41.161718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6b83-b401-4938-92ee-bebfe4fa477a service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:41.161989 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6b83-b401-4938-92ee-bebfe4fa477a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:41.179040 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b6b83-b401-4938-92ee-bebfe4fa477a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:41.180628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b6b83-b401-4938-92ee-bebfe4fa477a service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:41.190363 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 798/1601] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:41 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:41.194137 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552be6d-11e9-4e65-b021-895d7ca807d4 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:41.195538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 804/1602] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:41.208799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552be6d-11e9-4e65-b021-895d7ca807d4 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:41.215200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2d507-90bb-4be1-964a-f89bbd337fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:41.217139 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d507-90bb-4be1-964a-f89bbd337fec admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:41.217605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d507-90bb-4be1-964a-f89bbd337fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:41.218050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d507-90bb-4be1-964a-f89bbd337fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:41.219389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2d507-90bb-4be1-964a-f89bbd337fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:41.222925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2d507-90bb-4be1-964a-f89bbd337fec admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:41.242444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 799/1603] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:06:41.246516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b6b83-b401-4938-92ee-bebfe4fa477a service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:41.259500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b6b83-b401-4938-92ee-bebfe4fa477a service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:41.285320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531681f-0100-4f0d-8c1a-ef8f06f04cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:41.286007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531681f-0100-4f0d-8c1a-ef8f06f04cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:41.286675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531681f-0100-4f0d-8c1a-ef8f06f04cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:41.287017 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531681f-0100-4f0d-8c1a-ef8f06f04cc1 admin admin] PATH_INFO: `/v3/projects/ed35368fcaa741e0b0b769b93d126f17/users/07abd7cfc70a49a597d686f49c88d25f/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:41.305268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531681f-0100-4f0d-8c1a-ef8f06f04cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed35368fcaa741e0b0b769b93d126f17, user_id=07abd7cfc70a49a597d686f49c88d25f, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:41.320475 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531681f-0100-4f0d-8c1a-ef8f06f04cc1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:41.347397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 805/1604] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:41 2022] PUT /identity/v3/projects/ed35368fcaa741e0b0b769b93d126f17/users/07abd7cfc70a49a597d686f49c88d25f/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:41.356249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2d507-90bb-4be1-964a-f89bbd337fec admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:41.391629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2d507-90bb-4be1-964a-f89bbd337fec admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:41.528816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dae618-9163-4c8a-b462-7eb318b6b413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:41.529429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dae618-9163-4c8a-b462-7eb318b6b413 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:41.529706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dae618-9163-4c8a-b462-7eb318b6b413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:41.529977 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dae618-9163-4c8a-b462-7eb318b6b413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:41.530630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dae618-9163-4c8a-b462-7eb318b6b413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:41.532248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dae618-9163-4c8a-b462-7eb318b6b413 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:41.556300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 800/1605] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:41.560424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531681f-0100-4f0d-8c1a-ef8f06f04cc1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:41.579363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531681f-0100-4f0d-8c1a-ef8f06f04cc1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:41.584349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c25a6-af1e-4077-bb2b-e19516179fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:41.585338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c25a6-af1e-4077-bb2b-e19516179fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:41.585338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c25a6-af1e-4077-bb2b-e19516179fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:41.585756 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c25a6-af1e-4077-bb2b-e19516179fc9 admin admin] PATH_INFO: `/v3/projects/ed35368fcaa741e0b0b769b93d126f17/users/07abd7cfc70a49a597d686f49c88d25f/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:41.589310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c25a6-af1e-4077-bb2b-e19516179fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed35368fcaa741e0b0b769b93d126f17, user_id=07abd7cfc70a49a597d686f49c88d25f, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:41.596313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c25a6-af1e-4077-bb2b-e19516179fc9 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:41.634116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 806/1606] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:41 2022] PUT /identity/v3/projects/ed35368fcaa741e0b0b769b93d126f17/users/07abd7cfc70a49a597d686f49c88d25f/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:41.647078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dae618-9163-4c8a-b462-7eb318b6b413 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:41.648867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e7dae618-9163-4c8a-b462-7eb318b6b413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:41.663227 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dae618-9163-4c8a-b462-7eb318b6b413 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:41.696064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:41.757879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:41.759321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:41.760171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:41.790312 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] PATH_INFO: `/v3/projects/5d0a984b904a4641a4b300289b40aa87` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:41.792561 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d0a984b904a4641a4b300289b40aa87)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:41.796658 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:41.879437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d0a984b904a4641a4b300289b40aa87', 'request_id': 'req-096f8584-1090-4ff9-b15e-3ff233dcc330'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:41.886781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d0a984b904a4641a4b300289b40aa87', 'request_id': 'req-096f8584-1090-4ff9-b15e-3ff233dcc330'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:41.921572 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d0a984b904a4641a4b300289b40aa87', 'request_id': 'req-096f8584-1090-4ff9-b15e-3ff233dcc330'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:41.940293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] The token cache is being invalidate because project 5d0a984b904a4641a4b300289b40aa87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:41.940946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:41.941794 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:41.943938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 801/1607] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:41 2022] DELETE /identity/v3/projects/5d0a984b904a4641a4b300289b40aa87 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:41.947642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c25a6-af1e-4077-bb2b-e19516179fc9 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:41.949529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ba5c25a6-af1e-4077-bb2b-e19516179fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:41.966626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c25a6-af1e-4077-bb2b-e19516179fc9 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:42.054779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:42.055546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:42.055824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:42.056167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] PATH_INFO: `/v3/users/b012356b5703446e83ec1e44520aae24` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:42.057371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b012356b5703446e83ec1e44520aae24)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:42.059574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:42.121740 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b012356b5703446e83ec1e44520aae24'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.122467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b012356b5703446e83ec1e44520aae24'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.136590 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:42.148788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b012356b5703446e83ec1e44520aae24'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.174096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b012356b5703446e83ec1e44520aae24'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.175682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-5ab454ee-ddf0-4709-ba6c-1264a5998602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b012356b5703446e83ec1e44520aae24'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.198055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 807/1608] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:41 2022] DELETE /identity/v3/users/b012356b5703446e83ec1e44520aae24 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:42.569227 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:42.583838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8584-1090-4ff9-b15e-3ff233dcc330 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:42.684850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:42.686526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:42.686811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:42.687110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] PATH_INFO: `/v3/projects/250e97b9047a41e6b6530175ef5f9e28` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:42.688395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=250e97b9047a41e6b6530175ef5f9e28)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:42.690438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:42.755696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250e97b9047a41e6b6530175ef5f9e28', 'request_id': 'req-2f7ce02b-acfb-4468-a128-fdab166f02ff'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.764550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250e97b9047a41e6b6530175ef5f9e28', 'request_id': 'req-2f7ce02b-acfb-4468-a128-fdab166f02ff'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.766661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:42.779103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250e97b9047a41e6b6530175ef5f9e28', 'request_id': 'req-2f7ce02b-acfb-4468-a128-fdab166f02ff'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.795380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] The token cache is being invalidate because project 250e97b9047a41e6b6530175ef5f9e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:42.795680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.797493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-2f7ce02b-acfb-4468-a128-fdab166f02ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:42.799164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 802/1609] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:42 2022] DELETE /identity/v3/projects/250e97b9047a41e6b6530175ef5f9e28 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:42.804028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81198514-e891-4215-be4c-fea5c12decd4 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:42.804350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81198514-e891-4215-be4c-fea5c12decd4 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:42.804678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81198514-e891-4215-be4c-fea5c12decd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:42.838400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-81198514-e891-4215-be4c-fea5c12decd4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:42.839711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-81198514-e891-4215-be4c-fea5c12decd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:42.904038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8b5be-e716-4ff8-a6d4-9c4afe540cf8 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:42.905367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8b5be-e716-4ff8-a6d4-9c4afe540cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:42.905929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8b5be-e716-4ff8-a6d4-9c4afe540cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:42.918035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 808/1610] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:42.926768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aa825-6a2e-4e16-86a9-1ddb2e66f407 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:42.926877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aa825-6a2e-4e16-86a9-1ddb2e66f407 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:42.927149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aa825-6a2e-4e16-86a9-1ddb2e66f407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:42.948789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-b96aa825-6a2e-4e16-86a9-1ddb2e66f407 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:42.949495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-80d8b5be-e716-4ff8-a6d4-9c4afe540cf8 None None] MFA Rules not processed for user `07abd7cfc70a49a597d686f49c88d25f`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:42.950035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b96aa825-6a2e-4e16-86a9-1ddb2e66f407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:42.950654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-80d8b5be-e716-4ff8-a6d4-9c4afe540cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:42.963135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 809/1611] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:42.964244 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aa825-6a2e-4e16-86a9-1ddb2e66f407 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:42.977565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aa825-6a2e-4e16-86a9-1ddb2e66f407 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:42.982767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c508592-aeaf-4424-8eca-936d0635f173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:42.983925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c508592-aeaf-4424-8eca-936d0635f173 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:42.984343 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c508592-aeaf-4424-8eca-936d0635f173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:42.984803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c508592-aeaf-4424-8eca-936d0635f173 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:42.985646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c508592-aeaf-4424-8eca-936d0635f173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:42.986243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c508592-aeaf-4424-8eca-936d0635f173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:42.988085 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c508592-aeaf-4424-8eca-936d0635f173 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.004766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 810/1612] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:43.009464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c508592-aeaf-4424-8eca-936d0635f173 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.020512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c508592-aeaf-4424-8eca-936d0635f173 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.025840 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cffd4-2764-41da-beb4-da46356f04da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.026611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cffd4-2764-41da-beb4-da46356f04da admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.026890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cffd4-2764-41da-beb4-da46356f04da admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.027311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cffd4-2764-41da-beb4-da46356f04da admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.028074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cffd4-2764-41da-beb4-da46356f04da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.029847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cffd4-2764-41da-beb4-da46356f04da admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.059108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 803/1613] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:06:43.072876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8b5be-e716-4ff8-a6d4-9c4afe540cf8 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.076844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-80d8b5be-e716-4ff8-a6d4-9c4afe540cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:43.081826 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 811/1614] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:43 2022] POST /identity/v3/projects => generated 367 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:43.083033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cffd4-2764-41da-beb4-da46356f04da admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.162905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cffd4-2764-41da-beb4-da46356f04da admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.168175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512f75d-0ae6-4eaa-ab34-d62c8bb7755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.168860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8b5be-e716-4ff8-a6d4-9c4afe540cf8 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.169317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512f75d-0ae6-4eaa-ab34-d62c8bb7755e admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.169943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512f75d-0ae6-4eaa-ab34-d62c8bb7755e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.170413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512f75d-0ae6-4eaa-ab34-d62c8bb7755e admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.171212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512f75d-0ae6-4eaa-ab34-d62c8bb7755e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.173522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512f75d-0ae6-4eaa-ab34-d62c8bb7755e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.208745 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 812/1615] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:43 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:43.212506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512f75d-0ae6-4eaa-ab34-d62c8bb7755e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.224032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512f75d-0ae6-4eaa-ab34-d62c8bb7755e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.229802 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d68b10-90d8-4f27-a654-df379db21220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.230402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d68b10-90d8-4f27-a654-df379db21220 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.230675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d68b10-90d8-4f27-a654-df379db21220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.230944 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d68b10-90d8-4f27-a654-df379db21220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.231665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d68b10-90d8-4f27-a654-df379db21220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.233278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d68b10-90d8-4f27-a654-df379db21220 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.234090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b827c-16e4-4485-aed6-fc3dd24872d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.234718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b827c-16e4-4485-aed6-fc3dd24872d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.235010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b827c-16e4-4485-aed6-fc3dd24872d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.235316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b827c-16e4-4485-aed6-fc3dd24872d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.237068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-db2b827c-16e4-4485-aed6-fc3dd24872d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:43.244173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 813/1616] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:43.248298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d68b10-90d8-4f27-a654-df379db21220 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.258505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d68b10-90d8-4f27-a654-df379db21220 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.266701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2c4e5-3637-43e5-9308-5352d53f1cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.267898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2c4e5-3637-43e5-9308-5352d53f1cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.268169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2c4e5-3637-43e5-9308-5352d53f1cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.268431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2c4e5-3637-43e5-9308-5352d53f1cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.269054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2c4e5-3637-43e5-9308-5352d53f1cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.270770 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2c4e5-3637-43e5-9308-5352d53f1cbc admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.272862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:43.276798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:43.284136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 814/1617] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:43.288036 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2c4e5-3637-43e5-9308-5352d53f1cbc admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.297747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2c4e5-3637-43e5-9308-5352d53f1cbc admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.299927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b827c-16e4-4485-aed6-fc3dd24872d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.301396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b827c-16e4-4485-aed6-fc3dd24872d8 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.302511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064ca64-bad7-46ed-8fd7-a2c5fccefc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.303183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064ca64-bad7-46ed-8fd7-a2c5fccefc8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.303465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064ca64-bad7-46ed-8fd7-a2c5fccefc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.303734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064ca64-bad7-46ed-8fd7-a2c5fccefc8a admin admin] PATH_INFO: `/v3/projects/5a07317ebf954d178833a260c330f4dc/users/a13e812e0038473cbf9607313966c4a0/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.310455 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 804/1618] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:06:43.318916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064ca64-bad7-46ed-8fd7-a2c5fccefc8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a07317ebf954d178833a260c330f4dc, user_id=a13e812e0038473cbf9607313966c4a0, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.321623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064ca64-bad7-46ed-8fd7-a2c5fccefc8a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.335669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 815/1619] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:43 2022] PUT /identity/v3/projects/5a07317ebf954d178833a260c330f4dc/users/a13e812e0038473cbf9607313966c4a0/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:43.346963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab48e2-1863-4e67-9f48-f23687aacad8 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.347351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab48e2-1863-4e67-9f48-f23687aacad8 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.347625 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab48e2-1863-4e67-9f48-f23687aacad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.376041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-8eab48e2-1863-4e67-9f48-f23687aacad8 None None] MFA Rules not processed for user `a13e812e0038473cbf9607313966c4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:43.377900 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8eab48e2-1863-4e67-9f48-f23687aacad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:43.442201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 805/1620] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:43.447045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064ca64-bad7-46ed-8fd7-a2c5fccefc8a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.515804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064ca64-bad7-46ed-8fd7-a2c5fccefc8a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.521338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794c54f-03f9-49aa-a5cb-dad2f9e806d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.522011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794c54f-03f9-49aa-a5cb-dad2f9e806d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.522318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794c54f-03f9-49aa-a5cb-dad2f9e806d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.522591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794c54f-03f9-49aa-a5cb-dad2f9e806d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.523361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794c54f-03f9-49aa-a5cb-dad2f9e806d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.525296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794c54f-03f9-49aa-a5cb-dad2f9e806d2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.559531 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 816/1621] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:43 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:43.563621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab48e2-1863-4e67-9f48-f23687aacad8 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.579024 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab48e2-1863-4e67-9f48-f23687aacad8 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.724725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44c2d1-3f2f-4cd7-8c1f-a917d54025cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.725347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44c2d1-3f2f-4cd7-8c1f-a917d54025cb admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.725643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44c2d1-3f2f-4cd7-8c1f-a917d54025cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.725984 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44c2d1-3f2f-4cd7-8c1f-a917d54025cb admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.726690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44c2d1-3f2f-4cd7-8c1f-a917d54025cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.728723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44c2d1-3f2f-4cd7-8c1f-a917d54025cb admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.752384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 806/1622] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:43 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:43.756277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794c54f-03f9-49aa-a5cb-dad2f9e806d2 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.767217 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794c54f-03f9-49aa-a5cb-dad2f9e806d2 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.771778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f47ef6-ec37-4d68-8e54-7cd89eff7f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.772379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f47ef6-ec37-4d68-8e54-7cd89eff7f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.772655 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f47ef6-ec37-4d68-8e54-7cd89eff7f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.772925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f47ef6-ec37-4d68-8e54-7cd89eff7f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.773560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f47ef6-ec37-4d68-8e54-7cd89eff7f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.775173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f47ef6-ec37-4d68-8e54-7cd89eff7f3b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.786326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 817/1623] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:43.791441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44c2d1-3f2f-4cd7-8c1f-a917d54025cb admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.802166 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44c2d1-3f2f-4cd7-8c1f-a917d54025cb admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.806740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc77e2-6936-42a7-b688-51234a67cf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.807423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc77e2-6936-42a7-b688-51234a67cf8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.807700 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc77e2-6936-42a7-b688-51234a67cf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.807971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc77e2-6936-42a7-b688-51234a67cf8a admin admin] PATH_INFO: `/v3/projects/5ea3cace6c7942139d59a57f0fc74202/users/b2b72bcbce014d848721c27ec5d55966/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.822342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc77e2-6936-42a7-b688-51234a67cf8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea3cace6c7942139d59a57f0fc74202, user_id=b2b72bcbce014d848721c27ec5d55966, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.825018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc77e2-6936-42a7-b688-51234a67cf8a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.838269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 807/1624] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:43 2022] PUT /identity/v3/projects/5ea3cace6c7942139d59a57f0fc74202/users/b2b72bcbce014d848721c27ec5d55966/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:43.842203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f47ef6-ec37-4d68-8e54-7cd89eff7f3b admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.852464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f47ef6-ec37-4d68-8e54-7cd89eff7f3b admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.909501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b67ca-1565-4e2d-8ab7-63bbedbda13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.910174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b67ca-1565-4e2d-8ab7-63bbedbda13d admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.910462 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b67ca-1565-4e2d-8ab7-63bbedbda13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.910738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b67ca-1565-4e2d-8ab7-63bbedbda13d admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.911419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b67ca-1565-4e2d-8ab7-63bbedbda13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.913104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b67ca-1565-4e2d-8ab7-63bbedbda13d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.923206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 818/1625] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:06:43.927948 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc77e2-6936-42a7-b688-51234a67cf8a admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:43.938274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc77e2-6936-42a7-b688-51234a67cf8a admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:43.943500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fa066-5aea-4b87-bb45-ef4772545f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:43.943843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fa066-5aea-4b87-bb45-ef4772545f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.944125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fa066-5aea-4b87-bb45-ef4772545f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.944399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fa066-5aea-4b87-bb45-ef4772545f9d admin admin] PATH_INFO: `/v3/projects/5ea3cace6c7942139d59a57f0fc74202/users/b2b72bcbce014d848721c27ec5d55966/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:43.946296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392fa066-5aea-4b87-bb45-ef4772545f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea3cace6c7942139d59a57f0fc74202, user_id=b2b72bcbce014d848721c27ec5d55966, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:43.948986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392fa066-5aea-4b87-bb45-ef4772545f9d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:43.960288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 808/1626] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:43 2022] PUT /identity/v3/projects/5ea3cace6c7942139d59a57f0fc74202/users/b2b72bcbce014d848721c27ec5d55966/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:43.971231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ffd01-96ec-4825-a611-fc8d9559a2f3 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:43.971707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ffd01-96ec-4825-a611-fc8d9559a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:43.971796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ffd01-96ec-4825-a611-fc8d9559a2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:44.002551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-167ffd01-96ec-4825-a611-fc8d9559a2f3 None None] MFA Rules not processed for user `b2b72bcbce014d848721c27ec5d55966`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:44.003841 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-167ffd01-96ec-4825-a611-fc8d9559a2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:44.036297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:44.074650 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 819/1627] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:44.086302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fa066-5aea-4b87-bb45-ef4772545f9d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:44.108512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:44.144180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fa066-5aea-4b87-bb45-ef4772545f9d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:44.147797 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ca8ca-d104-4291-91e4-9ae51abf2495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:44.148401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ca8ca-d104-4291-91e4-9ae51abf2495 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:44.148676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ca8ca-d104-4291-91e4-9ae51abf2495 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:44.148950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ca8ca-d104-4291-91e4-9ae51abf2495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:44.159126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ca8ca-d104-4291-91e4-9ae51abf2495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:44.160581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ca8ca-d104-4291-91e4-9ae51abf2495 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:44.170146 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 809/1628] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:44.252824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ffd01-96ec-4825-a611-fc8d9559a2f3 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:44.262632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ffd01-96ec-4825-a611-fc8d9559a2f3 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:44.266218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb3664-4c19-4b27-bd67-4eca6f4093bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:44.266814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb3664-4c19-4b27-bd67-4eca6f4093bd service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:44.267640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb3664-4c19-4b27-bd67-4eca6f4093bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:44.268062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb3664-4c19-4b27-bd67-4eca6f4093bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:44.327179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb3664-4c19-4b27-bd67-4eca6f4093bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:44.328628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb3664-4c19-4b27-bd67-4eca6f4093bd service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:44.338128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 820/1629] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:49.027702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ca8ca-d104-4291-91e4-9ae51abf2495 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:49.029550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-746ca8ca-d104-4291-91e4-9ae51abf2495 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:49.040875 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ca8ca-d104-4291-91e4-9ae51abf2495 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:49.044522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08250d-058d-4cf4-ba82-b7d705412556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:49.045154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08250d-058d-4cf4-ba82-b7d705412556 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:49.045431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08250d-058d-4cf4-ba82-b7d705412556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:49.045701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08250d-058d-4cf4-ba82-b7d705412556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:49.131663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08250d-058d-4cf4-ba82-b7d705412556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:49.133033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08250d-058d-4cf4-ba82-b7d705412556 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:49.144834 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 810/1630] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:54.334542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=42 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:56.785525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb3664-4c19-4b27-bd67-4eca6f4093bd service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:56.917722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb3664-4c19-4b27-bd67-4eca6f4093bd service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:56.923220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:56.924180 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:56.925000 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:56.925602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] PATH_INFO: `/v3/users/a13e812e0038473cbf9607313966c4a0` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:56.927541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13e812e0038473cbf9607313966c4a0)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:56.930232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:57.010627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13e812e0038473cbf9607313966c4a0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.012151 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13e812e0038473cbf9607313966c4a0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.052410 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13e812e0038473cbf9607313966c4a0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.074208 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13e812e0038473cbf9607313966c4a0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.077216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13e812e0038473cbf9607313966c4a0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.109021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 821/1631] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:56 2022] DELETE /identity/v3/users/a13e812e0038473cbf9607313966c4a0 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:57.114565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6d09e-24cb-43a2-8dc6-0243ac43cac1 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:57.115033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6d09e-24cb-43a2-8dc6-0243ac43cac1 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:57.115231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6d09e-24cb-43a2-8dc6-0243ac43cac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:57.151327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-11d6d09e-24cb-43a2-8dc6-0243ac43cac1 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:57.153713 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-11d6d09e-24cb-43a2-8dc6-0243ac43cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:57.301018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 811/1632] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:57.313367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:57.316271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:57.408389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045438b-6168-4008-961b-6c6ce1f6c785 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:57.412546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7eb64-fff3-4318-8664-fdad1034b632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:57.413159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7eb64-fff3-4318-8664-fdad1034b632 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:57.413439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7eb64-fff3-4318-8664-fdad1034b632 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:57.413710 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7eb64-fff3-4318-8664-fdad1034b632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:57.427187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7eb64-fff3-4318-8664-fdad1034b632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:57.428639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7eb64-fff3-4318-8664-fdad1034b632 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:57.449978 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 822/1633] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:06:57.825171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6d09e-24cb-43a2-8dc6-0243ac43cac1 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:57.826584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-11d6d09e-24cb-43a2-8dc6-0243ac43cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:57.838609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6d09e-24cb-43a2-8dc6-0243ac43cac1 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:57.843258 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:57.843889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:57.844166 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:57.844431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] PATH_INFO: `/v3/projects/5a07317ebf954d178833a260c330f4dc` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:57.845580 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a07317ebf954d178833a260c330f4dc)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:57.847498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:57.908559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a07317ebf954d178833a260c330f4dc', 'request_id': 'req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.910407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:57.915468 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a07317ebf954d178833a260c330f4dc', 'request_id': 'req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.923941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a07317ebf954d178833a260c330f4dc', 'request_id': 'req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.934842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] The token cache is being invalidate because project 5a07317ebf954d178833a260c330f4dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:57.935175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.935800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:57.937384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 812/1634] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:57 2022] DELETE /identity/v3/projects/5a07317ebf954d178833a260c330f4dc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:57.941884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7eb64-fff3-4318-8664-fdad1034b632 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:57.951355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7eb64-fff3-4318-8664-fdad1034b632 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:57.969046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:58.022410 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:58.023124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.023400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.023667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] PATH_INFO: `/v3/users/b2b72bcbce014d848721c27ec5d55966` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.024775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b72bcbce014d848721c27ec5d55966)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:58.026687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:58.073984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b72bcbce014d848721c27ec5d55966'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.074637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b72bcbce014d848721c27ec5d55966'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.086637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b72bcbce014d848721c27ec5d55966'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.093618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b72bcbce014d848721c27ec5d55966'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.094271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-22ccdc3c-3fed-44c9-9b26-10295e4a81ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b72bcbce014d848721c27ec5d55966'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.107278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 823/1635] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:06:57 2022] DELETE /identity/v3/users/b2b72bcbce014d848721c27ec5d55966 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:58.371744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:58.373181 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:58.389710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf248f4e-ee46-4870-a2c5-5e0dd880ee1f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:58.452821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:58.453451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.453726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.454517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] PATH_INFO: `/v3/projects/5ea3cace6c7942139d59a57f0fc74202` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.455689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ea3cace6c7942139d59a57f0fc74202)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:58.457603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:58.506945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ea3cace6c7942139d59a57f0fc74202', 'request_id': 'req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.515321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ea3cace6c7942139d59a57f0fc74202', 'request_id': 'req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.523735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ea3cace6c7942139d59a57f0fc74202', 'request_id': 'req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.535882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] The token cache is being invalidate because project 5ea3cace6c7942139d59a57f0fc74202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:06:58.536178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.536854 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c4828b5f-ad4f-4bd4-ba1d-b6291e2e2e2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:06:58.538757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 813/1636] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:06:58 2022] DELETE /identity/v3/projects/5ea3cace6c7942139d59a57f0fc74202 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:58.543598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac2b20-dd6f-488e-b13b-34e02284e95a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.543883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac2b20-dd6f-488e-b13b-34e02284e95a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.544159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac2b20-dd6f-488e-b13b-34e02284e95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.563474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-92ac2b20-dd6f-488e-b13b-34e02284e95a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:58.564669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-92ac2b20-dd6f-488e-b13b-34e02284e95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:58.639414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 824/1637] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:58.644907 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9d919-730b-4810-9b8d-c2aaa892d976 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.645041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9d919-730b-4810-9b8d-c2aaa892d976 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.645359 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9d919-730b-4810-9b8d-c2aaa892d976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.677165 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-73a9d919-730b-4810-9b8d-c2aaa892d976 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:58.678362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-73a9d919-730b-4810-9b8d-c2aaa892d976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:58.689568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 814/1638] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:58.692169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac2b20-dd6f-488e-b13b-34e02284e95a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:58.702828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac2b20-dd6f-488e-b13b-34e02284e95a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:58.707937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fe524-282a-49aa-b1aa-126ba898d93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:58.708681 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fe524-282a-49aa-b1aa-126ba898d93e admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.708963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fe524-282a-49aa-b1aa-126ba898d93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.709232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fe524-282a-49aa-b1aa-126ba898d93e admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.709868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fe524-282a-49aa-b1aa-126ba898d93e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:06:58.710164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fe524-282a-49aa-b1aa-126ba898d93e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:58.711806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fe524-282a-49aa-b1aa-126ba898d93e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:58.723667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 825/1639] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:06:58.728038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9d919-730b-4810-9b8d-c2aaa892d976 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:58.737276 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9d919-730b-4810-9b8d-c2aaa892d976 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:58.741847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b4a75-c52d-4149-8847-e877d6fe044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:58.742440 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b4a75-c52d-4149-8847-e877d6fe044b admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.742717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b4a75-c52d-4149-8847-e877d6fe044b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.743006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b4a75-c52d-4149-8847-e877d6fe044b admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.743677 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b4a75-c52d-4149-8847-e877d6fe044b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:58.745622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b4a75-c52d-4149-8847-e877d6fe044b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:58.774726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 815/1640] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:58 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:06:58.779026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fe524-282a-49aa-b1aa-126ba898d93e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:58.788987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fe524-282a-49aa-b1aa-126ba898d93e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:58.814730 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:58.847604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e886aea-dece-432c-a18d-773ad526b5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:58.848206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e886aea-dece-432c-a18d-773ad526b5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.848482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e886aea-dece-432c-a18d-773ad526b5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.848753 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e886aea-dece-432c-a18d-773ad526b5e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.849371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e886aea-dece-432c-a18d-773ad526b5e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:58.851309 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e886aea-dece-432c-a18d-773ad526b5e6 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:58.871492 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 826/1641] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:58 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:58.875524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b4a75-c52d-4149-8847-e877d6fe044b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:58.886158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b4a75-c52d-4149-8847-e877d6fe044b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:58.891009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a87db0-7911-40ae-97e4-e4daf3b6f4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:58.892151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a87db0-7911-40ae-97e4-e4daf3b6f4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.892431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a87db0-7911-40ae-97e4-e4daf3b6f4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.892698 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a87db0-7911-40ae-97e4-e4daf3b6f4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.893339 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a87db0-7911-40ae-97e4-e4daf3b6f4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:58.894918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a87db0-7911-40ae-97e4-e4daf3b6f4a7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:58.904897 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 816/1642] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:58.908920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e886aea-dece-432c-a18d-773ad526b5e6 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:58.919623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e886aea-dece-432c-a18d-773ad526b5e6 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:58.924317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a914ca9-1d43-4ac4-9a23-b8ae09b1c924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:58.924906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a914ca9-1d43-4ac4-9a23-b8ae09b1c924 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.925182 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a914ca9-1d43-4ac4-9a23-b8ae09b1c924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.925533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a914ca9-1d43-4ac4-9a23-b8ae09b1c924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.926168 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a914ca9-1d43-4ac4-9a23-b8ae09b1c924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:58.927810 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a914ca9-1d43-4ac4-9a23-b8ae09b1c924 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:58.937814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 827/1643] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:06:58.942224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a87db0-7911-40ae-97e4-e4daf3b6f4a7 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:58.952139 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a87db0-7911-40ae-97e4-e4daf3b6f4a7 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:58.957321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34ce33-70b6-4d4c-adde-c39f8ae3e5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:58.957959 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34ce33-70b6-4d4c-adde-c39f8ae3e5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:58.958236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34ce33-70b6-4d4c-adde-c39f8ae3e5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:58.958525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34ce33-70b6-4d4c-adde-c39f8ae3e5d7 admin admin] PATH_INFO: `/v3/projects/685721cff16e4642a7928487c7fe919f/users/14ae5b0f79964dbca464b6c235a0cfa4/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:58.972726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34ce33-70b6-4d4c-adde-c39f8ae3e5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=685721cff16e4642a7928487c7fe919f, user_id=14ae5b0f79964dbca464b6c235a0cfa4, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:58.975420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34ce33-70b6-4d4c-adde-c39f8ae3e5d7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:58.989669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 817/1644] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:58 2022] PUT /identity/v3/projects/685721cff16e4642a7928487c7fe919f/users/14ae5b0f79964dbca464b6c235a0cfa4/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:59.001513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e971c-1d5b-4eb5-b912-6df35eeb70ff None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:59.002012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e971c-1d5b-4eb5-b912-6df35eeb70ff None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:59.002107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e971c-1d5b-4eb5-b912-6df35eeb70ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:59.030033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ec3e971c-1d5b-4eb5-b912-6df35eeb70ff None None] MFA Rules not processed for user `14ae5b0f79964dbca464b6c235a0cfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:59.031248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ec3e971c-1d5b-4eb5-b912-6df35eeb70ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:59.092854 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 828/1645] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:59.097248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34ce33-70b6-4d4c-adde-c39f8ae3e5d7 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:59.105925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:59.161087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34ce33-70b6-4d4c-adde-c39f8ae3e5d7 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:59.165693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06381e0-31a7-4e09-b86b-3b1555fae348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:59.166294 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06381e0-31a7-4e09-b86b-3b1555fae348 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:59.166574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06381e0-31a7-4e09-b86b-3b1555fae348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:59.166844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06381e0-31a7-4e09-b86b-3b1555fae348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:59.167498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06381e0-31a7-4e09-b86b-3b1555fae348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:59.169410 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06381e0-31a7-4e09-b86b-3b1555fae348 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:59.196421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 818/1646] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:06:59 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:59.200455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e971c-1d5b-4eb5-b912-6df35eeb70ff None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:59.210786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e971c-1d5b-4eb5-b912-6df35eeb70ff None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:59.265705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e204853-6a51-4b3f-8174-6f2ec0b98909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:59.266289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e204853-6a51-4b3f-8174-6f2ec0b98909 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:59.266612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e204853-6a51-4b3f-8174-6f2ec0b98909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:59.266882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e204853-6a51-4b3f-8174-6f2ec0b98909 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:59.267523 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e204853-6a51-4b3f-8174-6f2ec0b98909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:59.269422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e204853-6a51-4b3f-8174-6f2ec0b98909 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:59.291323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 829/1647] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:06:59 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:06:59.293641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06381e0-31a7-4e09-b86b-3b1555fae348 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:59.304620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06381e0-31a7-4e09-b86b-3b1555fae348 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:59.309271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1720c6-d88d-4413-806a-2de821eb6750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:59.309893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1720c6-d88d-4413-806a-2de821eb6750 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:59.310170 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1720c6-d88d-4413-806a-2de821eb6750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:59.310466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1720c6-d88d-4413-806a-2de821eb6750 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:59.311131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1720c6-d88d-4413-806a-2de821eb6750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:59.312809 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1720c6-d88d-4413-806a-2de821eb6750 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:59.323921 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 819/1648] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:59.327915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e204853-6a51-4b3f-8174-6f2ec0b98909 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:59.337898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e204853-6a51-4b3f-8174-6f2ec0b98909 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:59.342447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e360a-98b1-4824-b5a7-809d3fd220bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:59.343178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e360a-98b1-4824-b5a7-809d3fd220bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:59.343402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e360a-98b1-4824-b5a7-809d3fd220bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:59.343675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e360a-98b1-4824-b5a7-809d3fd220bf admin admin] PATH_INFO: `/v3/projects/98d127da8ad647cba917f62fe11542bf/users/c52279426c154252810eab9ab6298b29/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:59.359446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e360a-98b1-4824-b5a7-809d3fd220bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d127da8ad647cba917f62fe11542bf, user_id=c52279426c154252810eab9ab6298b29, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:59.362725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e360a-98b1-4824-b5a7-809d3fd220bf admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:59.376876 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 830/1649] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:59 2022] PUT /identity/v3/projects/98d127da8ad647cba917f62fe11542bf/users/c52279426c154252810eab9ab6298b29/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:06:59.381825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1720c6-d88d-4413-806a-2de821eb6750 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:59.392382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1720c6-d88d-4413-806a-2de821eb6750 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:59.448192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e29f9-498f-4e4c-b2dc-c81a7add1fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:59.448791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e29f9-498f-4e4c-b2dc-c81a7add1fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:59.449065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e29f9-498f-4e4c-b2dc-c81a7add1fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:59.449791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e29f9-498f-4e4c-b2dc-c81a7add1fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:59.449950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e29f9-498f-4e4c-b2dc-c81a7add1fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:59.451553 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e29f9-498f-4e4c-b2dc-c81a7add1fb8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:59.461391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 820/1650] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:06:59.466296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e360a-98b1-4824-b5a7-809d3fd220bf admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:59.472660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:06:59.478898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e360a-98b1-4824-b5a7-809d3fd220bf admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:59.483799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8f34-d708-4a4b-8104-bc9d475d7c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:59.484472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8f34-d708-4a4b-8104-bc9d475d7c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:59.484749 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8f34-d708-4a4b-8104-bc9d475d7c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:59.485017 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8f34-d708-4a4b-8104-bc9d475d7c79 admin admin] PATH_INFO: `/v3/projects/98d127da8ad647cba917f62fe11542bf/users/c52279426c154252810eab9ab6298b29/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:59.487558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db8f34-d708-4a4b-8104-bc9d475d7c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d127da8ad647cba917f62fe11542bf, user_id=c52279426c154252810eab9ab6298b29, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:59.490189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db8f34-d708-4a4b-8104-bc9d475d7c79 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:59.500331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 831/1651] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:06:59 2022] PUT /identity/v3/projects/98d127da8ad647cba917f62fe11542bf/users/c52279426c154252810eab9ab6298b29/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:06:59.510499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18752cc1-c1fd-4a82-9aa4-863c34c29f98 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:59.510783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18752cc1-c1fd-4a82-9aa4-863c34c29f98 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:59.511089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18752cc1-c1fd-4a82-9aa4-863c34c29f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:59.538728 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-18752cc1-c1fd-4a82-9aa4-863c34c29f98 None None] MFA Rules not processed for user `c52279426c154252810eab9ab6298b29`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:06:59.540011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-18752cc1-c1fd-4a82-9aa4-863c34c29f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:59.609059 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 821/1652] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:06:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:06:59.620425 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8f34-d708-4a4b-8104-bc9d475d7c79 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:06:59.622171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e9db8f34-d708-4a4b-8104-bc9d475d7c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:06:59.684850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8f34-d708-4a4b-8104-bc9d475d7c79 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:06:59.688681 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68446be6-ec45-4ce3-bbf8-4baa461dcc89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:06:59.689328 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68446be6-ec45-4ce3-bbf8-4baa461dcc89 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:06:59.689607 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68446be6-ec45-4ce3-bbf8-4baa461dcc89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:06:59.689877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68446be6-ec45-4ce3-bbf8-4baa461dcc89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:06:59.702755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68446be6-ec45-4ce3-bbf8-4baa461dcc89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:06:59.704307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68446be6-ec45-4ce3-bbf8-4baa461dcc89 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:06:59.719107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 832/1653] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:06:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:00.307271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18752cc1-c1fd-4a82-9aa4-863c34c29f98 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:00.311417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:00.372004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18752cc1-c1fd-4a82-9aa4-863c34c29f98 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:00.377120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:00.377792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:00.378018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:00.378290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] PATH_INFO: `/v3/users/14ae5b0f79964dbca464b6c235a0cfa4` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:00.379429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ae5b0f79964dbca464b6c235a0cfa4)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:00.381369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:00.425914 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ae5b0f79964dbca464b6c235a0cfa4'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:00.426557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ae5b0f79964dbca464b6c235a0cfa4'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:00.438155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ae5b0f79964dbca464b6c235a0cfa4'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:00.445384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ae5b0f79964dbca464b6c235a0cfa4'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:00.451604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ae5b0f79964dbca464b6c235a0cfa4'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:00.457970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 822/1654] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:00 2022] DELETE /identity/v3/users/14ae5b0f79964dbca464b6c235a0cfa4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:00.462684 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedd06b-7efa-490c-be97-33e65c57b60d None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:00.463203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedd06b-7efa-490c-be97-33e65c57b60d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:00.463203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedd06b-7efa-490c-be97-33e65c57b60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:00.482138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2dedd06b-7efa-490c-be97-33e65c57b60d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:00.483399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2dedd06b-7efa-490c-be97-33e65c57b60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:00.541013 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 833/1655] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:00.550855 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:00.552377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:00.607599 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ada77-0ef8-4659-b721-b363b3bd1a41 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:00.611712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d45096-314d-47c7-a8ec-0ce16831d109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:00.612300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d45096-314d-47c7-a8ec-0ce16831d109 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:00.612576 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d45096-314d-47c7-a8ec-0ce16831d109 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:00.612843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d45096-314d-47c7-a8ec-0ce16831d109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:00.621521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d45096-314d-47c7-a8ec-0ce16831d109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:00.622920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d45096-314d-47c7-a8ec-0ce16831d109 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:00.632884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 823/1656] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:00.921902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedd06b-7efa-490c-be97-33e65c57b60d None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:00.924150 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2dedd06b-7efa-490c-be97-33e65c57b60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:00.936248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedd06b-7efa-490c-be97-33e65c57b60d None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:00.941585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:00.942258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:00.942537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:00.942810 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] PATH_INFO: `/v3/projects/685721cff16e4642a7928487c7fe919f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:00.944010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=685721cff16e4642a7928487c7fe919f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:00.946107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:00.998387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '685721cff16e4642a7928487c7fe919f', 'request_id': 'req-2d3d24b7-5ee4-4c31-832a-bce14edb0284'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.006143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '685721cff16e4642a7928487c7fe919f', 'request_id': 'req-2d3d24b7-5ee4-4c31-832a-bce14edb0284'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.013080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '685721cff16e4642a7928487c7fe919f', 'request_id': 'req-2d3d24b7-5ee4-4c31-832a-bce14edb0284'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.024004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] The token cache is being invalidate because project 685721cff16e4642a7928487c7fe919f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:01.024289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.024966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.026445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 834/1657] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:00 2022] DELETE /identity/v3/projects/685721cff16e4642a7928487c7fe919f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:01.030502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d45096-314d-47c7-a8ec-0ce16831d109 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:01.040349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d45096-314d-47c7-a8ec-0ce16831d109 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:01.061460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:01.100366 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:01.100986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:01.101267 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:01.101538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] PATH_INFO: `/v3/users/c52279426c154252810eab9ab6298b29` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:01.102629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c52279426c154252810eab9ab6298b29)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:01.104565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:01.151816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c52279426c154252810eab9ab6298b29'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.152384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c52279426c154252810eab9ab6298b29'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.164589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c52279426c154252810eab9ab6298b29'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.171387 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c52279426c154252810eab9ab6298b29'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.179299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-4f4bb31f-1f87-43cf-ae53-d0ffd0fdf1e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c52279426c154252810eab9ab6298b29'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.185375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 824/1658] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:01 2022] DELETE /identity/v3/users/c52279426c154252810eab9ab6298b29 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:01.465038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:01.467709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:01.484886 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d24b7-5ee4-4c31-832a-bce14edb0284 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:01.502176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:01.560265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:01.560901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:01.561183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:01.561455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] PATH_INFO: `/v3/projects/98d127da8ad647cba917f62fe11542bf` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:01.562571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d127da8ad647cba917f62fe11542bf)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:01.564639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:01.609735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d127da8ad647cba917f62fe11542bf', 'request_id': 'req-a26ea71b-e01e-4bb7-8789-f07664a419d0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.616627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d127da8ad647cba917f62fe11542bf', 'request_id': 'req-a26ea71b-e01e-4bb7-8789-f07664a419d0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.622860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d127da8ad647cba917f62fe11542bf', 'request_id': 'req-a26ea71b-e01e-4bb7-8789-f07664a419d0'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.633161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] The token cache is being invalidate because project 98d127da8ad647cba917f62fe11542bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:01.633447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.634227 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a26ea71b-e01e-4bb7-8789-f07664a419d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:01.635775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 835/1659] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:01 2022] DELETE /identity/v3/projects/98d127da8ad647cba917f62fe11542bf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:01.640916 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d3ff0-3409-4af5-b60a-04ad6f80946a None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:01.641899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d3ff0-3409-4af5-b60a-04ad6f80946a None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:01.642095 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d3ff0-3409-4af5-b60a-04ad6f80946a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:01.662372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f03d3ff0-3409-4af5-b60a-04ad6f80946a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:01.663574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f03d3ff0-3409-4af5-b60a-04ad6f80946a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:01.722996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 825/1660] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:01.731301 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8170-1565-4398-a2db-96d45a0c0e6a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:01.731589 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8170-1565-4398-a2db-96d45a0c0e6a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:01.731906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8170-1565-4398-a2db-96d45a0c0e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:01.752699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-344c8170-1565-4398-a2db-96d45a0c0e6a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:01.754115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-344c8170-1565-4398-a2db-96d45a0c0e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:01.763213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 836/1661] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:01.768192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d3ff0-3409-4af5-b60a-04ad6f80946a None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:01.778568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d3ff0-3409-4af5-b60a-04ad6f80946a None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:01.783326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e3914-5146-4a55-9bcf-ffb5188bb294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:01.783918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e3914-5146-4a55-9bcf-ffb5188bb294 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:01.784193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e3914-5146-4a55-9bcf-ffb5188bb294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:01.784462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e3914-5146-4a55-9bcf-ffb5188bb294 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:01.785099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e3914-5146-4a55-9bcf-ffb5188bb294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:01.785395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e3914-5146-4a55-9bcf-ffb5188bb294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:01.787176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e3914-5146-4a55-9bcf-ffb5188bb294 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:01.803861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 826/1662] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:07:01.804930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c8170-1565-4398-a2db-96d45a0c0e6a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:01.818706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c8170-1565-4398-a2db-96d45a0c0e6a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:01.823636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e14cb-929c-42fa-8531-3eb2eb9e0609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:01.824247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e14cb-929c-42fa-8531-3eb2eb9e0609 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:01.824529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e14cb-929c-42fa-8531-3eb2eb9e0609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:01.824807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e14cb-929c-42fa-8531-3eb2eb9e0609 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:01.825448 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e14cb-929c-42fa-8531-3eb2eb9e0609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:01.827420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e14cb-929c-42fa-8531-3eb2eb9e0609 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:01.859637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 837/1663] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:01 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:01.860833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e3914-5146-4a55-9bcf-ffb5188bb294 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:01.871116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e3914-5146-4a55-9bcf-ffb5188bb294 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:01.903608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:01.945493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ac815-3716-46ca-8bad-164eb06f39c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:01.946647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ac815-3716-46ca-8bad-164eb06f39c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:01.946930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ac815-3716-46ca-8bad-164eb06f39c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:01.947373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ac815-3716-46ca-8bad-164eb06f39c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:01.948149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ac815-3716-46ca-8bad-164eb06f39c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:01.950795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ac815-3716-46ca-8bad-164eb06f39c3 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:01.990206 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 827/1664] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:01 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:01.995849 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e14cb-929c-42fa-8531-3eb2eb9e0609 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:02.008347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e14cb-929c-42fa-8531-3eb2eb9e0609 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:02.014560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13153e-7509-4411-9bac-28260195ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:02.015279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13153e-7509-4411-9bac-28260195ce71 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:02.015800 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13153e-7509-4411-9bac-28260195ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:02.016145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13153e-7509-4411-9bac-28260195ce71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:02.018892 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13153e-7509-4411-9bac-28260195ce71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:02.021098 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13153e-7509-4411-9bac-28260195ce71 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:02.031516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 838/1665] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:02.035828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ac815-3716-46ca-8bad-164eb06f39c3 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:02.047354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ac815-3716-46ca-8bad-164eb06f39c3 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:02.052972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc0dca-9cf1-49e8-bb79-6d2b5fb2f7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:02.053725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc0dca-9cf1-49e8-bb79-6d2b5fb2f7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:02.054070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc0dca-9cf1-49e8-bb79-6d2b5fb2f7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:02.054412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc0dca-9cf1-49e8-bb79-6d2b5fb2f7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:02.055313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc0dca-9cf1-49e8-bb79-6d2b5fb2f7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:02.057958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc0dca-9cf1-49e8-bb79-6d2b5fb2f7d4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:02.074890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 828/1666] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:02.081022 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13153e-7509-4411-9bac-28260195ce71 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:02.109000 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13153e-7509-4411-9bac-28260195ce71 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:02.122060 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3157e-bfae-4452-bd43-05570ca2535f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:02.122887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3157e-bfae-4452-bd43-05570ca2535f admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:02.123199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3157e-bfae-4452-bd43-05570ca2535f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:02.123557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3157e-bfae-4452-bd43-05570ca2535f admin admin] PATH_INFO: `/v3/projects/75e77eeb227b4d0499037e4e54d7c859/users/a88d979ca60b4e95841891cf36d7c06c/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:02.168670 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3157e-bfae-4452-bd43-05570ca2535f admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e77eeb227b4d0499037e4e54d7c859, user_id=a88d979ca60b4e95841891cf36d7c06c, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:02.171830 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3157e-bfae-4452-bd43-05570ca2535f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:02.195363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 839/1667] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:02 2022] PUT /identity/v3/projects/75e77eeb227b4d0499037e4e54d7c859/users/a88d979ca60b4e95841891cf36d7c06c/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:02.205959 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f1c14-fd12-4348-882d-3c515181b70d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:02.206248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f1c14-fd12-4348-882d-3c515181b70d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:02.206606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f1c14-fd12-4348-882d-3c515181b70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:02.252564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-295f1c14-fd12-4348-882d-3c515181b70d None None] MFA Rules not processed for user `a88d979ca60b4e95841891cf36d7c06c`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:02.254245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-295f1c14-fd12-4348-882d-3c515181b70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:02.364557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 829/1668] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:02.376704 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3157e-bfae-4452-bd43-05570ca2535f admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:02.378536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-34b3157e-bfae-4452-bd43-05570ca2535f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:02.429428 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:02.442390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3157e-bfae-4452-bd43-05570ca2535f admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:02.447138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202dc91-fa53-4086-8ad4-c0361947674c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:02.448012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202dc91-fa53-4086-8ad4-c0361947674c service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:02.448293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202dc91-fa53-4086-8ad4-c0361947674c service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:02.448565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202dc91-fa53-4086-8ad4-c0361947674c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:02.458877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202dc91-fa53-4086-8ad4-c0361947674c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:02.460418 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202dc91-fa53-4086-8ad4-c0361947674c service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:02.470700 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 840/1669] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:02.495066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f1c14-fd12-4348-882d-3c515181b70d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:02.556783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f1c14-fd12-4348-882d-3c515181b70d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:02.562016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:02.562656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:02.563090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:02.563405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] PATH_INFO: `/v3/users/a88d979ca60b4e95841891cf36d7c06c` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:02.564682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=a88d979ca60b4e95841891cf36d7c06c)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:02.568333 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:02.613221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:02.620049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a88d979ca60b4e95841891cf36d7c06c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:02.621214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a88d979ca60b4e95841891cf36d7c06c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:02.634198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a88d979ca60b4e95841891cf36d7c06c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:02.641453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a88d979ca60b4e95841891cf36d7c06c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:02.651009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a88d979ca60b4e95841891cf36d7c06c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:02.658082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 830/1670] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:02 2022] DELETE /identity/v3/users/a88d979ca60b4e95841891cf36d7c06c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:02.662727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbb8ad-e2b7-49de-88ca-c71a84cdf9d8 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:02.663262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbb8ad-e2b7-49de-88ca-c71a84cdf9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:02.663540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbb8ad-e2b7-49de-88ca-c71a84cdf9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:02.683299 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-55dbb8ad-e2b7-49de-88ca-c71a84cdf9d8 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:02.684449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-55dbb8ad-e2b7-49de-88ca-c71a84cdf9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:02.752103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 841/1671] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:02.762130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:02.763808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:02.835734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73788637-2cee-49b7-81eb-0caffe0b10ec admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:02.839320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078f712-b9b2-4686-ab45-9dc79b27579f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:02.839924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078f712-b9b2-4686-ab45-9dc79b27579f service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:02.840198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078f712-b9b2-4686-ab45-9dc79b27579f service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:02.840464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078f712-b9b2-4686-ab45-9dc79b27579f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:02.849760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078f712-b9b2-4686-ab45-9dc79b27579f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:02.851198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078f712-b9b2-4686-ab45-9dc79b27579f service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:02.860450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 831/1672] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:03.119189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbb8ad-e2b7-49de-88ca-c71a84cdf9d8 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:03.120656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-55dbb8ad-e2b7-49de-88ca-c71a84cdf9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:03.132378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbb8ad-e2b7-49de-88ca-c71a84cdf9d8 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:03.137313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:03.137980 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.138259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.138662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] PATH_INFO: `/v3/projects/75e77eeb227b4d0499037e4e54d7c859` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.139823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e77eeb227b4d0499037e4e54d7c859)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:03.142595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:03.197697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e77eeb227b4d0499037e4e54d7c859', 'request_id': 'req-d9af6813-9cc3-4657-821c-ef0a76fa40fd'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:03.204422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e77eeb227b4d0499037e4e54d7c859', 'request_id': 'req-d9af6813-9cc3-4657-821c-ef0a76fa40fd'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:03.211737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e77eeb227b4d0499037e4e54d7c859', 'request_id': 'req-d9af6813-9cc3-4657-821c-ef0a76fa40fd'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:03.222692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] The token cache is being invalidate because project 75e77eeb227b4d0499037e4e54d7c859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:03.223008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:03.223654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d9af6813-9cc3-4657-821c-ef0a76fa40fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:03.225465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 842/1673] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:03 2022] DELETE /identity/v3/projects/75e77eeb227b4d0499037e4e54d7c859 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:03.230280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dbb50-22bd-41e3-8de9-4f77490a89e4 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.230584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dbb50-22bd-41e3-8de9-4f77490a89e4 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.230927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dbb50-22bd-41e3-8de9-4f77490a89e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.253490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-920dbb50-22bd-41e3-8de9-4f77490a89e4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:03.254695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-920dbb50-22bd-41e3-8de9-4f77490a89e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:03.316542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 832/1674] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:03.324171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fb7c5-c371-4271-9646-411214ddd699 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.324459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fb7c5-c371-4271-9646-411214ddd699 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.324738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fb7c5-c371-4271-9646-411214ddd699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.344772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-345fb7c5-c371-4271-9646-411214ddd699 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:03.345958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-345fb7c5-c371-4271-9646-411214ddd699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:03.353742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 843/1675] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:03.357706 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dbb50-22bd-41e3-8de9-4f77490a89e4 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:03.367619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dbb50-22bd-41e3-8de9-4f77490a89e4 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:03.372335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87adaed-042b-4754-af2e-39bf18f67198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:03.372938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87adaed-042b-4754-af2e-39bf18f67198 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.373274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87adaed-042b-4754-af2e-39bf18f67198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.373596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87adaed-042b-4754-af2e-39bf18f67198 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.374244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87adaed-042b-4754-af2e-39bf18f67198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:03.374674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87adaed-042b-4754-af2e-39bf18f67198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:03.376881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87adaed-042b-4754-af2e-39bf18f67198 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:03.388976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 833/1676] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:07:03.393941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fb7c5-c371-4271-9646-411214ddd699 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:03.403203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fb7c5-c371-4271-9646-411214ddd699 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:03.407857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3b808-710a-4b82-81a7-e3ab60be3c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:03.408452 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3b808-710a-4b82-81a7-e3ab60be3c41 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.408728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3b808-710a-4b82-81a7-e3ab60be3c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.408998 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3b808-710a-4b82-81a7-e3ab60be3c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.409633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3b808-710a-4b82-81a7-e3ab60be3c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:03.411564 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3b808-710a-4b82-81a7-e3ab60be3c41 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:03.443241 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 844/1677] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:03 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:03.444394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87adaed-042b-4754-af2e-39bf18f67198 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:03.455072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87adaed-042b-4754-af2e-39bf18f67198 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:03.505856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:03.527055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e171a-0710-4bd5-8970-49d12369aee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:03.527892 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e171a-0710-4bd5-8970-49d12369aee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.528174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e171a-0710-4bd5-8970-49d12369aee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.528449 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e171a-0710-4bd5-8970-49d12369aee8 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.529099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e171a-0710-4bd5-8970-49d12369aee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:03.531298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e171a-0710-4bd5-8970-49d12369aee8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:03.557558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 834/1678] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:03 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:03.570319 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3b808-710a-4b82-81a7-e3ab60be3c41 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:03.582666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3b808-710a-4b82-81a7-e3ab60be3c41 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:03.587466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fa864-9d8f-4db1-b717-1670ede903fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:03.588088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fa864-9d8f-4db1-b717-1670ede903fc admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.588544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fa864-9d8f-4db1-b717-1670ede903fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.588821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fa864-9d8f-4db1-b717-1670ede903fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.589474 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fa864-9d8f-4db1-b717-1670ede903fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:03.591128 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fa864-9d8f-4db1-b717-1670ede903fc admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:03.602204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 845/1679] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:03.606457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e171a-0710-4bd5-8970-49d12369aee8 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:03.624991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e171a-0710-4bd5-8970-49d12369aee8 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:03.630317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a48a2-479a-4cfd-8493-e9f402ab3f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:03.630942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a48a2-479a-4cfd-8493-e9f402ab3f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.631254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a48a2-479a-4cfd-8493-e9f402ab3f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.631532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a48a2-479a-4cfd-8493-e9f402ab3f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.632188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a48a2-479a-4cfd-8493-e9f402ab3f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:03.633823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a48a2-479a-4cfd-8493-e9f402ab3f16 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:03.645500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 835/1680] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:03.649662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fa864-9d8f-4db1-b717-1670ede903fc admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:03.659979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fa864-9d8f-4db1-b717-1670ede903fc admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:03.664685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccc280-ddea-4545-8f50-1fbf74d15a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:03.665346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccc280-ddea-4545-8f50-1fbf74d15a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.665625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccc280-ddea-4545-8f50-1fbf74d15a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.665895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccc280-ddea-4545-8f50-1fbf74d15a96 admin admin] PATH_INFO: `/v3/projects/dcdb7cdbe9f44af2901b31624affba0b/users/f34bb1b4b91a4a4aab7aa632a734f053/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.682043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccc280-ddea-4545-8f50-1fbf74d15a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcdb7cdbe9f44af2901b31624affba0b, user_id=f34bb1b4b91a4a4aab7aa632a734f053, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:03.684733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccc280-ddea-4545-8f50-1fbf74d15a96 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:03.698733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 846/1681] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:03 2022] PUT /identity/v3/projects/dcdb7cdbe9f44af2901b31624affba0b/users/f34bb1b4b91a4a4aab7aa632a734f053/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:03.709554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fe7e5-1222-4b43-b6c9-6db14c891dda None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.709773 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fe7e5-1222-4b43-b6c9-6db14c891dda None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.710049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fe7e5-1222-4b43-b6c9-6db14c891dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.738105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-909fe7e5-1222-4b43-b6c9-6db14c891dda None None] MFA Rules not processed for user `f34bb1b4b91a4a4aab7aa632a734f053`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:03.739346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-909fe7e5-1222-4b43-b6c9-6db14c891dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:03.803308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 836/1682] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:03.807736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccc280-ddea-4545-8f50-1fbf74d15a96 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:03.813854 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:03.872559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccc280-ddea-4545-8f50-1fbf74d15a96 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:03.877852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c62f6-b38d-4fec-8952-d154d0f61eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:03.878456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c62f6-b38d-4fec-8952-d154d0f61eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.878738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c62f6-b38d-4fec-8952-d154d0f61eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.879039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c62f6-b38d-4fec-8952-d154d0f61eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.879674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c62f6-b38d-4fec-8952-d154d0f61eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:03.881586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c62f6-b38d-4fec-8952-d154d0f61eb0 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:03.907514 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 847/1683] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:03 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:03.911502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fe7e5-1222-4b43-b6c9-6db14c891dda None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:03.921858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fe7e5-1222-4b43-b6c9-6db14c891dda None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:03.976323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24f6be-db13-4252-9b26-90bdc22d1f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:03.976850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24f6be-db13-4252-9b26-90bdc22d1f3d admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:03.977127 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24f6be-db13-4252-9b26-90bdc22d1f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:03.977397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24f6be-db13-4252-9b26-90bdc22d1f3d admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:03.978039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24f6be-db13-4252-9b26-90bdc22d1f3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:03.979990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24f6be-db13-4252-9b26-90bdc22d1f3d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:04.000674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 837/1684] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:03 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:04.004495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c62f6-b38d-4fec-8952-d154d0f61eb0 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:04.014761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c62f6-b38d-4fec-8952-d154d0f61eb0 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:04.019374 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca4157-01d6-465a-a17d-682fcafb56db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:04.019962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca4157-01d6-465a-a17d-682fcafb56db admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:04.020237 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca4157-01d6-465a-a17d-682fcafb56db admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:04.020506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca4157-01d6-465a-a17d-682fcafb56db admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:04.021134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca4157-01d6-465a-a17d-682fcafb56db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:04.022708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca4157-01d6-465a-a17d-682fcafb56db admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:04.033413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 848/1685] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:04.037277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24f6be-db13-4252-9b26-90bdc22d1f3d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:04.047081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24f6be-db13-4252-9b26-90bdc22d1f3d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:04.052608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4328b9-703f-42f6-af8d-dd1ba0bb9e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:04.053243 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4328b9-703f-42f6-af8d-dd1ba0bb9e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:04.053525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4328b9-703f-42f6-af8d-dd1ba0bb9e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:04.053822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4328b9-703f-42f6-af8d-dd1ba0bb9e20 admin admin] PATH_INFO: `/v3/projects/c7099fb2013147ea83c1e2812a05d00e/users/b73ad5e2ee4747c9bf42342d061d7ac9/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:04.069006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4328b9-703f-42f6-af8d-dd1ba0bb9e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7099fb2013147ea83c1e2812a05d00e, user_id=b73ad5e2ee4747c9bf42342d061d7ac9, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:04.071731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4328b9-703f-42f6-af8d-dd1ba0bb9e20 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:04.084393 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 838/1686] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:04 2022] PUT /identity/v3/projects/c7099fb2013147ea83c1e2812a05d00e/users/b73ad5e2ee4747c9bf42342d061d7ac9/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:04.088299 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca4157-01d6-465a-a17d-682fcafb56db admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:04.098702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca4157-01d6-465a-a17d-682fcafb56db admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:04.152921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3ac78-8014-480a-8ff7-e220ec31a6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:04.153517 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3ac78-8014-480a-8ff7-e220ec31a6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:04.153797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3ac78-8014-480a-8ff7-e220ec31a6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:04.154065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3ac78-8014-480a-8ff7-e220ec31a6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:04.154693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3ac78-8014-480a-8ff7-e220ec31a6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:04.156310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3ac78-8014-480a-8ff7-e220ec31a6a7 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:04.166222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 849/1687] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:04.170320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4328b9-703f-42f6-af8d-dd1ba0bb9e20 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:04.180935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4328b9-703f-42f6-af8d-dd1ba0bb9e20 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:04.185477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd9142-af50-48e8-a928-f1c721369d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:04.186109 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd9142-af50-48e8-a928-f1c721369d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:04.186382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd9142-af50-48e8-a928-f1c721369d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:04.186648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd9142-af50-48e8-a928-f1c721369d8b admin admin] PATH_INFO: `/v3/projects/c7099fb2013147ea83c1e2812a05d00e/users/b73ad5e2ee4747c9bf42342d061d7ac9/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:04.189091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd9142-af50-48e8-a928-f1c721369d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7099fb2013147ea83c1e2812a05d00e, user_id=b73ad5e2ee4747c9bf42342d061d7ac9, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:04.191787 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd9142-af50-48e8-a928-f1c721369d8b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:04.202021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 839/1688] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:04 2022] PUT /identity/v3/projects/c7099fb2013147ea83c1e2812a05d00e/users/b73ad5e2ee4747c9bf42342d061d7ac9/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:04.212131 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1519e-3fe7-4138-8f69-7214fa2552bd None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:04.212422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1519e-3fe7-4138-8f69-7214fa2552bd None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:04.212698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1519e-3fe7-4138-8f69-7214fa2552bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:04.239689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-49f1519e-3fe7-4138-8f69-7214fa2552bd None None] MFA Rules not processed for user `b73ad5e2ee4747c9bf42342d061d7ac9`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:04.240806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-49f1519e-3fe7-4138-8f69-7214fa2552bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:04.306080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:04.311115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 850/1689] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:04.321824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd9142-af50-48e8-a928-f1c721369d8b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:04.323613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-08dd9142-af50-48e8-a928-f1c721369d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:04.341339 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:04.383668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd9142-af50-48e8-a928-f1c721369d8b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:04.387527 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bdd2f-aec6-4e37-a2cc-af6bbaf24f31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:04.388125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bdd2f-aec6-4e37-a2cc-af6bbaf24f31 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:04.388401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bdd2f-aec6-4e37-a2cc-af6bbaf24f31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:04.388668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bdd2f-aec6-4e37-a2cc-af6bbaf24f31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:04.446286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bdd2f-aec6-4e37-a2cc-af6bbaf24f31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:04.447763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bdd2f-aec6-4e37-a2cc-af6bbaf24f31 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:04.456966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 840/1690] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:11.310261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1519e-3fe7-4138-8f69-7214fa2552bd None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:11.373553 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1519e-3fe7-4138-8f69-7214fa2552bd None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:11.378828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:11.379546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:11.379823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:11.380091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] PATH_INFO: `/v3/users/f34bb1b4b91a4a4aab7aa632a734f053` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:11.381217 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f34bb1b4b91a4a4aab7aa632a734f053)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:11.383190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:11.432859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f34bb1b4b91a4a4aab7aa632a734f053'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:11.433683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f34bb1b4b91a4a4aab7aa632a734f053'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:11.447109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f34bb1b4b91a4a4aab7aa632a734f053'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:11.454035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f34bb1b4b91a4a4aab7aa632a734f053'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:11.454706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f34bb1b4b91a4a4aab7aa632a734f053'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:11.478902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 851/1691] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:11 2022] DELETE /identity/v3/users/f34bb1b4b91a4a4aab7aa632a734f053 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:11.486082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a193f8-cd93-47be-81d3-ce67d49e6d69 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:11.486541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a193f8-cd93-47be-81d3-ce67d49e6d69 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:11.486663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a193f8-cd93-47be-81d3-ce67d49e6d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:11.507516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-54a193f8-cd93-47be-81d3-ce67d49e6d69 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:11.508857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-54a193f8-cd93-47be-81d3-ce67d49e6d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:11.591506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 841/1692] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:11.605225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:11.606766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:11.679562 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06006271-ce52-4eb6-beb3-5ce9cb70d480 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:11.684106 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152685ed-d5cd-4efb-8fd3-02be1c3d411c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:11.684897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152685ed-d5cd-4efb-8fd3-02be1c3d411c service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:11.685183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152685ed-d5cd-4efb-8fd3-02be1c3d411c service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:11.685506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152685ed-d5cd-4efb-8fd3-02be1c3d411c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:11.698380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152685ed-d5cd-4efb-8fd3-02be1c3d411c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:11.699959 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152685ed-d5cd-4efb-8fd3-02be1c3d411c service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:11.710435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 852/1693] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:12.002499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a193f8-cd93-47be-81d3-ce67d49e6d69 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:12.003927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-54a193f8-cd93-47be-81d3-ce67d49e6d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:12.017100 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a193f8-cd93-47be-81d3-ce67d49e6d69 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:12.021756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:12.022381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:12.022660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:12.022936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] PATH_INFO: `/v3/projects/dcdb7cdbe9f44af2901b31624affba0b` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:12.024057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcdb7cdbe9f44af2901b31624affba0b)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:12.025951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:12.071310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcdb7cdbe9f44af2901b31624affba0b', 'request_id': 'req-27c8483d-db99-4718-894e-dd1538f29e62'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.077443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcdb7cdbe9f44af2901b31624affba0b', 'request_id': 'req-27c8483d-db99-4718-894e-dd1538f29e62'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.081661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:12.085721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcdb7cdbe9f44af2901b31624affba0b', 'request_id': 'req-27c8483d-db99-4718-894e-dd1538f29e62'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.099083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] The token cache is being invalidate because project dcdb7cdbe9f44af2901b31624affba0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:12.099365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.100085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.101717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 842/1694] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:12 2022] DELETE /identity/v3/projects/dcdb7cdbe9f44af2901b31624affba0b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:12.106872 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152685ed-d5cd-4efb-8fd3-02be1c3d411c service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:12.122577 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152685ed-d5cd-4efb-8fd3-02be1c3d411c service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:12.155276 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:12.239012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:12.240139 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:12.240698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:12.241099 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] PATH_INFO: `/v3/users/b73ad5e2ee4747c9bf42342d061d7ac9` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:12.242376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b73ad5e2ee4747c9bf42342d061d7ac9)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:12.244672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:12.310647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b73ad5e2ee4747c9bf42342d061d7ac9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.311801 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b73ad5e2ee4747c9bf42342d061d7ac9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.326824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b73ad5e2ee4747c9bf42342d061d7ac9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.337001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b73ad5e2ee4747c9bf42342d061d7ac9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.337961 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-e27a71b3-63cf-4081-b338-2081d7e1357f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b73ad5e2ee4747c9bf42342d061d7ac9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.357909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 853/1695] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:12 2022] DELETE /identity/v3/users/b73ad5e2ee4747c9bf42342d061d7ac9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:12.650573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:12.652104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:12.665052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8483d-db99-4718-894e-dd1538f29e62 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:12.725403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:12.726043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:12.726326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:12.726598 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] PATH_INFO: `/v3/projects/c7099fb2013147ea83c1e2812a05d00e` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:12.727755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7099fb2013147ea83c1e2812a05d00e)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:12.729672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:12.780603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7099fb2013147ea83c1e2812a05d00e', 'request_id': 'req-efed23b2-e7a9-4422-b7ae-0534a40f8142'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.787445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7099fb2013147ea83c1e2812a05d00e', 'request_id': 'req-efed23b2-e7a9-4422-b7ae-0534a40f8142'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.795866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7099fb2013147ea83c1e2812a05d00e', 'request_id': 'req-efed23b2-e7a9-4422-b7ae-0534a40f8142'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.807968 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] The token cache is being invalidate because project c7099fb2013147ea83c1e2812a05d00e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:12.808364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.808989 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efed23b2-e7a9-4422-b7ae-0534a40f8142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:12.810513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 843/1696] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:12 2022] DELETE /identity/v3/projects/c7099fb2013147ea83c1e2812a05d00e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:12.816218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a924ed-2e76-40b1-adb5-39bbf215b915 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:12.816533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a924ed-2e76-40b1-adb5-39bbf215b915 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:12.816804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a924ed-2e76-40b1-adb5-39bbf215b915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:12.836787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-76a924ed-2e76-40b1-adb5-39bbf215b915 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:12.838226 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-76a924ed-2e76-40b1-adb5-39bbf215b915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:12.905251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 854/1697] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:12.912805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a543b4-d924-437a-8209-3ac5a60795a8 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:12.913148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a543b4-d924-437a-8209-3ac5a60795a8 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:12.913423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a543b4-d924-437a-8209-3ac5a60795a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:12.939939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-43a543b4-d924-437a-8209-3ac5a60795a8 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:12.941142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-43a543b4-d924-437a-8209-3ac5a60795a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:12.950973 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 844/1698] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:12.956519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a924ed-2e76-40b1-adb5-39bbf215b915 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:12.968675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a924ed-2e76-40b1-adb5-39bbf215b915 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:12.975221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff2bc6-89fe-40e2-bf6d-d26c083ec6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:12.975944 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff2bc6-89fe-40e2-bf6d-d26c083ec6fd admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:12.976331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff2bc6-89fe-40e2-bf6d-d26c083ec6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:12.976727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff2bc6-89fe-40e2-bf6d-d26c083ec6fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:12.977783 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff2bc6-89fe-40e2-bf6d-d26c083ec6fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:12.977927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff2bc6-89fe-40e2-bf6d-d26c083ec6fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:12.979943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff2bc6-89fe-40e2-bf6d-d26c083ec6fd admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:12.998127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 855/1699] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:07:13.002785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a543b4-d924-437a-8209-3ac5a60795a8 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:13.014664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a543b4-d924-437a-8209-3ac5a60795a8 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:13.019607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae146dc-a601-4842-b3b1-158c3435b48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:13.020220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae146dc-a601-4842-b3b1-158c3435b48f admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:13.020681 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae146dc-a601-4842-b3b1-158c3435b48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:13.020958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae146dc-a601-4842-b3b1-158c3435b48f admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:13.021588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae146dc-a601-4842-b3b1-158c3435b48f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:13.023584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae146dc-a601-4842-b3b1-158c3435b48f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:13.063080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 845/1700] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:13 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:13.079011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff2bc6-89fe-40e2-bf6d-d26c083ec6fd admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:13.090027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff2bc6-89fe-40e2-bf6d-d26c083ec6fd admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:13.119885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:13.161276 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6efd3-d1a4-4379-9b15-c4dfa5409c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:13.162022 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6efd3-d1a4-4379-9b15-c4dfa5409c28 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:13.162350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6efd3-d1a4-4379-9b15-c4dfa5409c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:13.162682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6efd3-d1a4-4379-9b15-c4dfa5409c28 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:13.163430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6efd3-d1a4-4379-9b15-c4dfa5409c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:13.165453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6efd3-d1a4-4379-9b15-c4dfa5409c28 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:13.190952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 856/1701] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:13 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:13.195318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae146dc-a601-4842-b3b1-158c3435b48f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:13.223176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae146dc-a601-4842-b3b1-158c3435b48f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:13.229763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661b55f-783c-4b1d-853f-5c6840f330bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:13.230934 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661b55f-783c-4b1d-853f-5c6840f330bd admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:13.232025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661b55f-783c-4b1d-853f-5c6840f330bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:13.232486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661b55f-783c-4b1d-853f-5c6840f330bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:13.233571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661b55f-783c-4b1d-853f-5c6840f330bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:13.236892 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661b55f-783c-4b1d-853f-5c6840f330bd admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:13.254392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 846/1702] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:13.258347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6efd3-d1a4-4379-9b15-c4dfa5409c28 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:13.269987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6efd3-d1a4-4379-9b15-c4dfa5409c28 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:13.278639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f1e5e-4dd8-4701-ab95-a7b70388cfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:13.294268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1e5e-4dd8-4701-ab95-a7b70388cfe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:13.294553 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1e5e-4dd8-4701-ab95-a7b70388cfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:13.294829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1e5e-4dd8-4701-ab95-a7b70388cfe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:13.295548 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f1e5e-4dd8-4701-ab95-a7b70388cfe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:13.298240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f1e5e-4dd8-4701-ab95-a7b70388cfe2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:13.314589 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 857/1703] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:13.316909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661b55f-783c-4b1d-853f-5c6840f330bd admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:13.332068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661b55f-783c-4b1d-853f-5c6840f330bd admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:13.339151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397f14f-90ac-4125-b9ff-34fc693ba8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:13.340313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397f14f-90ac-4125-b9ff-34fc693ba8f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:13.340655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397f14f-90ac-4125-b9ff-34fc693ba8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:13.341045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397f14f-90ac-4125-b9ff-34fc693ba8f5 admin admin] PATH_INFO: `/v3/projects/aa4bbb7ba7f14bd9bcaaa0ea461322a6/users/fec54d8c1ea24cc8bbbeb6fef1fd2028/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:13.362417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397f14f-90ac-4125-b9ff-34fc693ba8f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4bbb7ba7f14bd9bcaaa0ea461322a6, user_id=fec54d8c1ea24cc8bbbeb6fef1fd2028, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:13.365925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397f14f-90ac-4125-b9ff-34fc693ba8f5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:13.387285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 847/1704] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:13 2022] PUT /identity/v3/projects/aa4bbb7ba7f14bd9bcaaa0ea461322a6/users/fec54d8c1ea24cc8bbbeb6fef1fd2028/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:13.400066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ca0ee-7645-4c97-955e-7d7e13ef2a1a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:13.402134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ca0ee-7645-4c97-955e-7d7e13ef2a1a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:13.403540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ca0ee-7645-4c97-955e-7d7e13ef2a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:13.459431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-3a4ca0ee-7645-4c97-955e-7d7e13ef2a1a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:13.463961 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3a4ca0ee-7645-4c97-955e-7d7e13ef2a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:13.561811 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 858/1705] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:13.576995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397f14f-90ac-4125-b9ff-34fc693ba8f5 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:13.579507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7397f14f-90ac-4125-b9ff-34fc693ba8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:13.595391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:13.648167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397f14f-90ac-4125-b9ff-34fc693ba8f5 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:13.652302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c3ec3-5009-43ba-9dd8-18c15f87720a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:13.653031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c3ec3-5009-43ba-9dd8-18c15f87720a service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:13.653468 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c3ec3-5009-43ba-9dd8-18c15f87720a service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:13.653840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c3ec3-5009-43ba-9dd8-18c15f87720a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:13.664196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c3ec3-5009-43ba-9dd8-18c15f87720a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:13.665774 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c3ec3-5009-43ba-9dd8-18c15f87720a service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:13.676688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 848/1706] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:14.110996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ca0ee-7645-4c97-955e-7d7e13ef2a1a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:14.112812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3a4ca0ee-7645-4c97-955e-7d7e13ef2a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:14.129599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ca0ee-7645-4c97-955e-7d7e13ef2a1a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:14.134828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:14.135633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:14.136016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:14.136390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] PATH_INFO: `/v3/users/30e6afd91dd94ad58ba94ae642149dae` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:14.137875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30e6afd91dd94ad58ba94ae642149dae)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:14.139975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:14.200736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30e6afd91dd94ad58ba94ae642149dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.201484 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30e6afd91dd94ad58ba94ae642149dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.214631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30e6afd91dd94ad58ba94ae642149dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.222870 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30e6afd91dd94ad58ba94ae642149dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.223622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30e6afd91dd94ad58ba94ae642149dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.237243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 859/1707] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:14 2022] DELETE /identity/v3/users/30e6afd91dd94ad58ba94ae642149dae => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:14.242017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfc0ac-9033-4d52-9162-b0cd052f8949 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:14.242374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfc0ac-9033-4d52-9162-b0cd052f8949 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:14.242721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfc0ac-9033-4d52-9162-b0cd052f8949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:14.262528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-25dfc0ac-9033-4d52-9162-b0cd052f8949 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:14.263878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-25dfc0ac-9033-4d52-9162-b0cd052f8949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:14.329328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 849/1708] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:14.339982 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:14.342360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:14.367229 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:14.407803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ff4ba-2389-49ca-94ef-6a6b5ff2ebf5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:14.411011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed11fd-7271-4346-96c7-2196ecbf19e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:14.411630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed11fd-7271-4346-96c7-2196ecbf19e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:14.411909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed11fd-7271-4346-96c7-2196ecbf19e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:14.412179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed11fd-7271-4346-96c7-2196ecbf19e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:14.421391 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed11fd-7271-4346-96c7-2196ecbf19e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:14.423426 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed11fd-7271-4346-96c7-2196ecbf19e4 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:14.433043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 860/1709] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:14.720778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfc0ac-9033-4d52-9162-b0cd052f8949 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:14.723579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-25dfc0ac-9033-4d52-9162-b0cd052f8949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:14.738871 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfc0ac-9033-4d52-9162-b0cd052f8949 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:14.744171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:14.744899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:14.745242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:14.745685 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] PATH_INFO: `/v3/users/4f033761116c4a36bf77f7eec4c469c6` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:14.750997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed11fd-7271-4346-96c7-2196ecbf19e4 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:14.753007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0aed11fd-7271-4346-96c7-2196ecbf19e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:14.762577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f033761116c4a36bf77f7eec4c469c6)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:14.764583 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:14.787458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed11fd-7271-4346-96c7-2196ecbf19e4 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:14.803184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:14.804172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:14.804743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:14.805313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] PATH_INFO: `/v3/projects/0b085d7cb8b2469ebb367df3049c3409` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:14.809432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b085d7cb8b2469ebb367df3049c3409)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:14.818049 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:14.873238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f033761116c4a36bf77f7eec4c469c6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.873961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f033761116c4a36bf77f7eec4c469c6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.908021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f033761116c4a36bf77f7eec4c469c6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.937237 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b085d7cb8b2469ebb367df3049c3409', 'request_id': 'req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.946195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f033761116c4a36bf77f7eec4c469c6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.959895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-54bbb5b6-9d16-4928-a9a3-06fa29ac4eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f033761116c4a36bf77f7eec4c469c6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.968738 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b085d7cb8b2469ebb367df3049c3409', 'request_id': 'req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.974230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 850/1710] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:14 2022] DELETE /identity/v3/users/4f033761116c4a36bf77f7eec4c469c6 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:14.978926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972ccc7-0cce-4cde-ad64-302db8076080 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:14.979234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972ccc7-0cce-4cde-ad64-302db8076080 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:14.979507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972ccc7-0cce-4cde-ad64-302db8076080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:14.980023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b085d7cb8b2469ebb367df3049c3409', 'request_id': 'req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.993324 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] The token cache is being invalidate because project 0b085d7cb8b2469ebb367df3049c3409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:14.993640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.994334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:14.996395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 861/1711] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:14 2022] DELETE /identity/v3/projects/0b085d7cb8b2469ebb367df3049c3409 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:15.000684 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:15.002344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:15.026764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4972ccc7-0cce-4cde-ad64-302db8076080 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:15.027998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4972ccc7-0cce-4cde-ad64-302db8076080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:15.082856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:15.159062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 851/1712] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:15.171290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972ccc7-0cce-4cde-ad64-302db8076080 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:15.172588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4972ccc7-0cce-4cde-ad64-302db8076080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:15.183143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff3674-d718-4e3d-b7e0-208dfdcb21ec admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:15.193916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:15.194855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:15.195077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:15.195364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] PATH_INFO: `/v3/users/8684018a3cae42499c2d607c92e42bf3` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:15.201755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8684018a3cae42499c2d607c92e42bf3)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:15.205077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:15.280577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972ccc7-0cce-4cde-ad64-302db8076080 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:15.288107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab82b4-4953-4496-9b7b-91d83dbfd054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:15.288880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab82b4-4953-4496-9b7b-91d83dbfd054 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:15.289170 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab82b4-4953-4496-9b7b-91d83dbfd054 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:15.289531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab82b4-4953-4496-9b7b-91d83dbfd054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:15.301577 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8684018a3cae42499c2d607c92e42bf3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:15.302858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8684018a3cae42499c2d607c92e42bf3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:15.304348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:15.321967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8684018a3cae42499c2d607c92e42bf3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:15.332443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab82b4-4953-4496-9b7b-91d83dbfd054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:15.333224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8684018a3cae42499c2d607c92e42bf3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:15.335367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8684018a3cae42499c2d607c92e42bf3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:15.336669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab82b4-4953-4496-9b7b-91d83dbfd054 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:15.378102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 862/1713] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:14 2022] DELETE /identity/v3/users/8684018a3cae42499c2d607c92e42bf3 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:15.381314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 852/1714] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:16.382021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab82b4-4953-4496-9b7b-91d83dbfd054 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:16.453704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab82b4-4953-4496-9b7b-91d83dbfd054 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:16.593939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:16.594601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:16.595053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:16.595346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] PATH_INFO: `/v3/projects/fbab451d31a94a239b3e442d8724d9e7` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:16.598395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbab451d31a94a239b3e442d8724d9e7)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:16.600286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:16.603442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:16.605660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:16.642877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c19782-00f9-4a02-82f3-cb3b2c5fa96a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:16.661821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:16.662498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:16.662776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:16.663077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] PATH_INFO: `/v3/projects/86d5fe15d7a64d17bec96264a8eba963` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:16.667817 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d5fe15d7a64d17bec96264a8eba963)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:16.669778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:16.689433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbab451d31a94a239b3e442d8724d9e7', 'request_id': 'req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.698843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbab451d31a94a239b3e442d8724d9e7', 'request_id': 'req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.710721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbab451d31a94a239b3e442d8724d9e7', 'request_id': 'req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.726539 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] The token cache is being invalidate because project fbab451d31a94a239b3e442d8724d9e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:16.727509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.728371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ce2d7e6c-da6d-4c85-8fc8-13083add45ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.730271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 853/1715] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:16 2022] DELETE /identity/v3/projects/fbab451d31a94a239b3e442d8724d9e7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:16.735414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040c11-5b0e-440d-a5e0-cd24bacbd7b8 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:16.740740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040c11-5b0e-440d-a5e0-cd24bacbd7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:16.741090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040c11-5b0e-440d-a5e0-cd24bacbd7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:16.762357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d5fe15d7a64d17bec96264a8eba963', 'request_id': 'req-7f4a4502-e78d-4eba-b3dd-129403147f22'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.774258 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:16.784262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d5fe15d7a64d17bec96264a8eba963', 'request_id': 'req-7f4a4502-e78d-4eba-b3dd-129403147f22'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.793643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d5fe15d7a64d17bec96264a8eba963', 'request_id': 'req-7f4a4502-e78d-4eba-b3dd-129403147f22'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.793819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-25040c11-5b0e-440d-a5e0-cd24bacbd7b8 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:16.795271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-25040c11-5b0e-440d-a5e0-cd24bacbd7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:16.812962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] The token cache is being invalidate because project 86d5fe15d7a64d17bec96264a8eba963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:16.813295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.814839 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7f4a4502-e78d-4eba-b3dd-129403147f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:16.824381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 863/1716] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:16 2022] DELETE /identity/v3/projects/86d5fe15d7a64d17bec96264a8eba963 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:16.825828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ec4c0-e613-4166-92d6-ec1c8176684d None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:16.826120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ec4c0-e613-4166-92d6-ec1c8176684d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:16.826634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ec4c0-e613-4166-92d6-ec1c8176684d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:16.856052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-095ec4c0-e613-4166-92d6-ec1c8176684d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:16.857708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-095ec4c0-e613-4166-92d6-ec1c8176684d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:16.892648 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 854/1717] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:16.901447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b6c4-9200-44f0-9e45-e8005d06d9c9 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:16.901759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b6c4-9200-44f0-9e45-e8005d06d9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:16.902000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b6c4-9200-44f0-9e45-e8005d06d9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:16.932250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-5dc8b6c4-9200-44f0-9e45-e8005d06d9c9 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:16.933403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5dc8b6c4-9200-44f0-9e45-e8005d06d9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:16.945221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 855/1718] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:16.949478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b6c4-9200-44f0-9e45-e8005d06d9c9 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:16.960966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 864/1719] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:16.962494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7084d9-4852-4ac5-95b7-9a1d4133c08b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:16.962777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7084d9-4852-4ac5-95b7-9a1d4133c08b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:16.963082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7084d9-4852-4ac5-95b7-9a1d4133c08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:16.963816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b6c4-9200-44f0-9e45-e8005d06d9c9 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:16.969830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9a5c1-a6dc-451e-b610-86ec2e6faca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:16.970647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9a5c1-a6dc-451e-b610-86ec2e6faca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:16.971104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9a5c1-a6dc-451e-b610-86ec2e6faca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:16.971526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9a5c1-a6dc-451e-b610-86ec2e6faca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:16.972450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9a5c1-a6dc-451e-b610-86ec2e6faca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:16.972905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9a5c1-a6dc-451e-b610-86ec2e6faca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:16.974679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9a5c1-a6dc-451e-b610-86ec2e6faca8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:16.986052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-dd7084d9-4852-4ac5-95b7-9a1d4133c08b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:16.987491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-dd7084d9-4852-4ac5-95b7-9a1d4133c08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:16.993697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 856/1720] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:07:16.994783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9a5c1-a6dc-451e-b610-86ec2e6faca8 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:16.997073 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 865/1721] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:17.001045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7084d9-4852-4ac5-95b7-9a1d4133c08b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.007712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9a5c1-a6dc-451e-b610-86ec2e6faca8 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.014015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7084d9-4852-4ac5-95b7-9a1d4133c08b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.015223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15fbf3-8a18-42ea-bbf5-a77d5946f5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.016355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15fbf3-8a18-42ea-bbf5-a77d5946f5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.016355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15fbf3-8a18-42ea-bbf5-a77d5946f5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.016557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15fbf3-8a18-42ea-bbf5-a77d5946f5a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.017391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15fbf3-8a18-42ea-bbf5-a77d5946f5a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.020599 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15fbf3-8a18-42ea-bbf5-a77d5946f5a2 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.020786 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ea415-29a8-4e9a-965a-770f8c5bda5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.022343 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ea415-29a8-4e9a-965a-770f8c5bda5c admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.022625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ea415-29a8-4e9a-965a-770f8c5bda5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.022893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ea415-29a8-4e9a-965a-770f8c5bda5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.023855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ea415-29a8-4e9a-965a-770f8c5bda5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:17.024161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ea415-29a8-4e9a-965a-770f8c5bda5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.025771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ea415-29a8-4e9a-965a-770f8c5bda5c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.049628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 866/1722] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:07:17.052646 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ea415-29a8-4e9a-965a-770f8c5bda5c admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.064740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 857/1723] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:16 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:17.065181 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ea415-29a8-4e9a-965a-770f8c5bda5c admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.069997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15fbf3-8a18-42ea-bbf5-a77d5946f5a2 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.096889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:17.159124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15fbf3-8a18-42ea-bbf5-a77d5946f5a2 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.167285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87d6fa-fd69-419d-8c2c-e16e661255c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.168514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87d6fa-fd69-419d-8c2c-e16e661255c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.168977 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87d6fa-fd69-419d-8c2c-e16e661255c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.169452 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87d6fa-fd69-419d-8c2c-e16e661255c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.170297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87d6fa-fd69-419d-8c2c-e16e661255c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.171257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0337000-5e40-4d06-8f1d-73347e0b33b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.172078 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0337000-5e40-4d06-8f1d-73347e0b33b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.172372 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0337000-5e40-4d06-8f1d-73347e0b33b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.172648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0337000-5e40-4d06-8f1d-73347e0b33b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.173301 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0337000-5e40-4d06-8f1d-73347e0b33b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.174554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87d6fa-fd69-419d-8c2c-e16e661255c0 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.175366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0337000-5e40-4d06-8f1d-73347e0b33b1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.216245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 858/1724] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:17 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:17.221234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87d6fa-fd69-419d-8c2c-e16e661255c0 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.235993 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87d6fa-fd69-419d-8c2c-e16e661255c0 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.241488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5cba8-ed5a-4eb2-a2f1-fc3ca29ca607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.242182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5cba8-ed5a-4eb2-a2f1-fc3ca29ca607 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.242538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5cba8-ed5a-4eb2-a2f1-fc3ca29ca607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.242883 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5cba8-ed5a-4eb2-a2f1-fc3ca29ca607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.243794 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5cba8-ed5a-4eb2-a2f1-fc3ca29ca607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.247361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5cba8-ed5a-4eb2-a2f1-fc3ca29ca607 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.248585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 867/1725] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:17 2022] POST /identity/v3/projects => generated 365 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:17.249709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0337000-5e40-4d06-8f1d-73347e0b33b1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.269295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 859/1726] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:17.272344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5cba8-ed5a-4eb2-a2f1-fc3ca29ca607 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.293291 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5cba8-ed5a-4eb2-a2f1-fc3ca29ca607 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.350912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0337000-5e40-4d06-8f1d-73347e0b33b1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.359251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee52a92-d426-4da5-b378-2ceacd4a35a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.360048 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee52a92-d426-4da5-b378-2ceacd4a35a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.360403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee52a92-d426-4da5-b378-2ceacd4a35a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.360744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee52a92-d426-4da5-b378-2ceacd4a35a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.361453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee52a92-d426-4da5-b378-2ceacd4a35a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.363682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee52a92-d426-4da5-b378-2ceacd4a35a8 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.367323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:17.401910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 868/1727] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:17 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:17.407233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee52a92-d426-4da5-b378-2ceacd4a35a8 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.411505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8623762-2777-4190-a431-8dbe614e88d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.412875 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8623762-2777-4190-a431-8dbe614e88d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.413395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8623762-2777-4190-a431-8dbe614e88d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.413718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8623762-2777-4190-a431-8dbe614e88d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.414571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8623762-2777-4190-a431-8dbe614e88d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.416784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8623762-2777-4190-a431-8dbe614e88d7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.421193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee52a92-d426-4da5-b378-2ceacd4a35a8 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.429791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7c771-b48c-40a2-8342-e87a3be6e633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.431379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7c771-b48c-40a2-8342-e87a3be6e633 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.431851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7c771-b48c-40a2-8342-e87a3be6e633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.432304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7c771-b48c-40a2-8342-e87a3be6e633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.432967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 860/1728] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:17.433210 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7c771-b48c-40a2-8342-e87a3be6e633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.435064 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7c771-b48c-40a2-8342-e87a3be6e633 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.438301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8623762-2777-4190-a431-8dbe614e88d7 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.448011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 869/1729] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:17.452588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7c771-b48c-40a2-8342-e87a3be6e633 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.461116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8623762-2777-4190-a431-8dbe614e88d7 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.466506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7c771-b48c-40a2-8342-e87a3be6e633 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.466817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dce1e-7b95-46ac-8409-a7d39980a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.467619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dce1e-7b95-46ac-8409-a7d39980a590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.467906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dce1e-7b95-46ac-8409-a7d39980a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.468178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dce1e-7b95-46ac-8409-a7d39980a590 admin admin] PATH_INFO: `/v3/projects/042b27b19bf34bdab59f3c1f6ac2c968/users/9ac573af98574997b4b026ba1ce50058/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.473691 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db3d06-a8ee-4920-b3e5-800d1c9a4f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.474365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db3d06-a8ee-4920-b3e5-800d1c9a4f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.474645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db3d06-a8ee-4920-b3e5-800d1c9a4f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.474919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db3d06-a8ee-4920-b3e5-800d1c9a4f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.475648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db3d06-a8ee-4920-b3e5-800d1c9a4f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.477441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db3d06-a8ee-4920-b3e5-800d1c9a4f77 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.492352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 870/1730] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:17.495585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db3d06-a8ee-4920-b3e5-800d1c9a4f77 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.495956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dce1e-7b95-46ac-8409-a7d39980a590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=042b27b19bf34bdab59f3c1f6ac2c968, user_id=9ac573af98574997b4b026ba1ce50058, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.500547 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dce1e-7b95-46ac-8409-a7d39980a590 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.520093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db3d06-a8ee-4920-b3e5-800d1c9a4f77 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:17.524843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 861/1731] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:17 2022] PUT /identity/v3/projects/042b27b19bf34bdab59f3c1f6ac2c968/users/9ac573af98574997b4b026ba1ce50058/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:17.536949 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5e623-d9cf-4c72-91ea-d3f3553a4ad5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.537317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5e623-d9cf-4c72-91ea-d3f3553a4ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.537804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5e623-d9cf-4c72-91ea-d3f3553a4ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.638314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f5d5e623-d9cf-4c72-91ea-d3f3553a4ad5 None None] MFA Rules not processed for user `9ac573af98574997b4b026ba1ce50058`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:17.641909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f5d5e623-d9cf-4c72-91ea-d3f3553a4ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:17.666183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff9d24-6e5f-460b-a2d3-9ff6602bbd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:17.667078 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff9d24-6e5f-460b-a2d3-9ff6602bbd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.667273 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff9d24-6e5f-460b-a2d3-9ff6602bbd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.667551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff9d24-6e5f-460b-a2d3-9ff6602bbd2f admin admin] PATH_INFO: `/v3/projects/ceb988fe6d614b6f860fb0a995a5444f/users/a1ecc4efe96c4d39ad88bb3138caff12/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.701798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff9d24-6e5f-460b-a2d3-9ff6602bbd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb988fe6d614b6f860fb0a995a5444f, user_id=a1ecc4efe96c4d39ad88bb3138caff12, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:17.705889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff9d24-6e5f-460b-a2d3-9ff6602bbd2f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:17.742628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 871/1732] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:17 2022] PUT /identity/v3/projects/ceb988fe6d614b6f860fb0a995a5444f/users/a1ecc4efe96c4d39ad88bb3138caff12/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:17.745270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e0c29-8c70-49e4-907a-e6dc28be4e9a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:17.745562 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e0c29-8c70-49e4-907a-e6dc28be4e9a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:17.745839 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e0c29-8c70-49e4-907a-e6dc28be4e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:17.768156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:17.845305 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-cf2e0c29-8c70-49e4-907a-e6dc28be4e9a None None] MFA Rules not processed for user `a1ecc4efe96c4d39ad88bb3138caff12`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:17.845441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 862/1733] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:17.850275 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5e623-d9cf-4c72-91ea-d3f3553a4ad5 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:17.851862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cf2e0c29-8c70-49e4-907a-e6dc28be4e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:17.876658 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5e623-d9cf-4c72-91ea-d3f3553a4ad5 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:18.016637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91755ab6-541b-42a5-b402-6f082b15ab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:18.019182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91755ab6-541b-42a5-b402-6f082b15ab4b admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:18.019471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91755ab6-541b-42a5-b402-6f082b15ab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:18.019751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91755ab6-541b-42a5-b402-6f082b15ab4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:18.024469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 872/1734] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:18.028402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91755ab6-541b-42a5-b402-6f082b15ab4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:18.035808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91755ab6-541b-42a5-b402-6f082b15ab4b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:18.041666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e0c29-8c70-49e4-907a-e6dc28be4e9a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:18.043925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cf2e0c29-8c70-49e4-907a-e6dc28be4e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:18.106280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 863/1735] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:17 2022] POST /identity/v3/projects => generated 381 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:18.107335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91755ab6-541b-42a5-b402-6f082b15ab4b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:18.182349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:18.257627 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91755ab6-541b-42a5-b402-6f082b15ab4b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:18.302844 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e0c29-8c70-49e4-907a-e6dc28be4e9a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:18.308564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb387fa-e4d9-4005-b034-d68bbb978437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:18.312267 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb387fa-e4d9-4005-b034-d68bbb978437 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:18.312597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb387fa-e4d9-4005-b034-d68bbb978437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:18.326230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb387fa-e4d9-4005-b034-d68bbb978437 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:18.326230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb387fa-e4d9-4005-b034-d68bbb978437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:18.326230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb387fa-e4d9-4005-b034-d68bbb978437 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:18.369341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 864/1736] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:18 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:18.373719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb387fa-e4d9-4005-b034-d68bbb978437 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:18.386137 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb387fa-e4d9-4005-b034-d68bbb978437 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:18.396426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a535f-8059-4a7e-813b-8672038666f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:18.397519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a535f-8059-4a7e-813b-8672038666f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:18.397806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a535f-8059-4a7e-813b-8672038666f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:18.398083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a535f-8059-4a7e-813b-8672038666f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:18.398743 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a535f-8059-4a7e-813b-8672038666f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:18.400403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a535f-8059-4a7e-813b-8672038666f8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:18.411083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261ca21-e551-41a4-9d74-84a474225b43 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8ec16bd9254b23bfd2b60551af9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:18.411954 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261ca21-e551-41a4-9d74-84a474225b43 service glance] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:18.412522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261ca21-e551-41a4-9d74-84a474225b43 service glance] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:18.413801 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261ca21-e551-41a4-9d74-84a474225b43 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:18.432335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 865/1737] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:18.433865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a535f-8059-4a7e-813b-8672038666f8 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:18.466676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a535f-8059-4a7e-813b-8672038666f8 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:18.477331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c7b4c-c16b-495d-88bc-4818b506bd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:18.478461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c7b4c-c16b-495d-88bc-4818b506bd53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:18.478939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c7b4c-c16b-495d-88bc-4818b506bd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:18.480027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:18.481447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c7b4c-c16b-495d-88bc-4818b506bd53 admin admin] PATH_INFO: `/v3/projects/5431eb1c3cab4b50a609a2d6ea3421c6/users/c1da771ca0634d7bbbe1de7e99f7dea8/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:18.498307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261ca21-e551-41a4-9d74-84a474225b43 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:18.500022 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261ca21-e551-41a4-9d74-84a474225b43 service glance] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:18.505138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c7b4c-c16b-495d-88bc-4818b506bd53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5431eb1c3cab4b50a609a2d6ea3421c6, user_id=c1da771ca0634d7bbbe1de7e99f7dea8, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:18.508139 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c7b4c-c16b-495d-88bc-4818b506bd53 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:18.523323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 873/1738] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:07:18.538036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 866/1739] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:18 2022] PUT /identity/v3/projects/5431eb1c3cab4b50a609a2d6ea3421c6/users/c1da771ca0634d7bbbe1de7e99f7dea8/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:18.542221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261ca21-e551-41a4-9d74-84a474225b43 service glance] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:18.562163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261ca21-e551-41a4-9d74-84a474225b43 service glance] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:18.673844 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85a42c-ba5c-42ff-9e94-9c4e3f467ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:18.674502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85a42c-ba5c-42ff-9e94-9c4e3f467ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:18.676270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85a42c-ba5c-42ff-9e94-9c4e3f467ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:18.676775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85a42c-ba5c-42ff-9e94-9c4e3f467ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:18.677549 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85a42c-ba5c-42ff-9e94-9c4e3f467ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:18.682917 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85a42c-ba5c-42ff-9e94-9c4e3f467ce8 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:18.701743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 874/1740] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:18.706965 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c7b4c-c16b-495d-88bc-4818b506bd53 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:18.732054 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c7b4c-c16b-495d-88bc-4818b506bd53 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:18.746799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e54fc3-6bb6-4337-b5c2-f09af9d94f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:18.747507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e54fc3-6bb6-4337-b5c2-f09af9d94f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:18.747798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e54fc3-6bb6-4337-b5c2-f09af9d94f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:18.748050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e54fc3-6bb6-4337-b5c2-f09af9d94f4d admin admin] PATH_INFO: `/v3/projects/5431eb1c3cab4b50a609a2d6ea3421c6/users/c1da771ca0634d7bbbe1de7e99f7dea8/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:18.750111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e54fc3-6bb6-4337-b5c2-f09af9d94f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5431eb1c3cab4b50a609a2d6ea3421c6, user_id=c1da771ca0634d7bbbe1de7e99f7dea8, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:18.761250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e54fc3-6bb6-4337-b5c2-f09af9d94f4d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:18.807187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 867/1741] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:18 2022] PUT /identity/v3/projects/5431eb1c3cab4b50a609a2d6ea3421c6/users/c1da771ca0634d7bbbe1de7e99f7dea8/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:18.819289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553079d2-e38f-4a6d-96c5-3a78735d8de5 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:18.819647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553079d2-e38f-4a6d-96c5-3a78735d8de5 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:18.819939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553079d2-e38f-4a6d-96c5-3a78735d8de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:18.875644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-553079d2-e38f-4a6d-96c5-3a78735d8de5 None None] MFA Rules not processed for user `c1da771ca0634d7bbbe1de7e99f7dea8`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:18.876878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-553079d2-e38f-4a6d-96c5-3a78735d8de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:19.052307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 875/1742] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:19.063994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e54fc3-6bb6-4337-b5c2-f09af9d94f4d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:19.067252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-91e54fc3-6bb6-4337-b5c2-f09af9d94f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:19.272177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e54fc3-6bb6-4337-b5c2-f09af9d94f4d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:19.279018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac69e04-3d0f-46a6-b1d3-9240755c5848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:19.279855 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac69e04-3d0f-46a6-b1d3-9240755c5848 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:19.280203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac69e04-3d0f-46a6-b1d3-9240755c5848 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:19.280546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac69e04-3d0f-46a6-b1d3-9240755c5848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:19.299162 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac69e04-3d0f-46a6-b1d3-9240755c5848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:19.301656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac69e04-3d0f-46a6-b1d3-9240755c5848 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:19.351256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 868/1743] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:19.519418 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f835cc3-92eb-44cd-a031-cce44e4e08f8 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:19.519742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f835cc3-92eb-44cd-a031-cce44e4e08f8 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:19.519980 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f835cc3-92eb-44cd-a031-cce44e4e08f8 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:19.521533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 876/1744] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:07:19.542836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadbb6b-0e2e-44c9-a940-08b51208a6f2 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:19.543890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadbb6b-0e2e-44c9-a940-08b51208a6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:19.544367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadbb6b-0e2e-44c9-a940-08b51208a6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:19.611510 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-7cadbb6b-0e2e-44c9-a940-08b51208a6f2 None None] MFA Rules not processed for user `da958657ac3041848d24a76b76705489`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:19.618272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7cadbb6b-0e2e-44c9-a940-08b51208a6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:19.691908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f835cc3-92eb-44cd-a031-cce44e4e08f8 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:19.698497 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9f835cc3-92eb-44cd-a031-cce44e4e08f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:19.877553 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f835cc3-92eb-44cd-a031-cce44e4e08f8 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:19.879283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 869/1745] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:07:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:19.888354 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:19.889307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:19.889740 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:19.890164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] PATH_INFO: `/v3/users/9bf29df38e1f4d3ab23d93321e8f491d` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:19.892212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf29df38e1f4d3ab23d93321e8f491d)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:19.893640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc86e8-401c-4f5e-93cd-1604bc529f16 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:19.894039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc86e8-401c-4f5e-93cd-1604bc529f16 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:19.894201 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc86e8-401c-4f5e-93cd-1604bc529f16 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:19.895321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:19.895864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 870/1746] 10.222.0.58 () {54 vars in 961 bytes} [Fri Oct 28 19:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 19:07:19.902267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:19.903061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6700f-05aa-47f2-89f3-50e391ab3b10 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:19.903388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6700f-05aa-47f2-89f3-50e391ab3b10 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:19.904409 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6700f-05aa-47f2-89f3-50e391ab3b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:19.943826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:19.963830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2ec6700f-05aa-47f2-89f3-50e391ab3b10 None None] MFA Rules not processed for user `da958657ac3041848d24a76b76705489`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:19.965404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2ec6700f-05aa-47f2-89f3-50e391ab3b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:20.013391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 871/1747] 10.222.0.58 () {58 vars in 1058 bytes} [Fri Oct 28 19:07:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:20.016333 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6700f-05aa-47f2-89f3-50e391ab3b10 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:20.026771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf29df38e1f4d3ab23d93321e8f491d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.031030 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf29df38e1f4d3ab23d93321e8f491d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.043860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6700f-05aa-47f2-89f3-50e391ab3b10 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:20.069304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf29df38e1f4d3ab23d93321e8f491d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.074535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb366cb-fa27-4f44-8f7b-2cf9f9f1f4cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da958657ac3041848d24a76b76705489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:20.077370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb366cb-fa27-4f44-8f7b-2cf9f9f1f4cb service placement] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:20.077812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb366cb-fa27-4f44-8f7b-2cf9f9f1f4cb service placement] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:20.078860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb366cb-fa27-4f44-8f7b-2cf9f9f1f4cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:20.085519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6bb366cb-fa27-4f44-8f7b-2cf9f9f1f4cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:20.100100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf29df38e1f4d3ab23d93321e8f491d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.102456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf29df38e1f4d3ab23d93321e8f491d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.124053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 877/1748] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:19 2022] DELETE /identity/v3/users/9bf29df38e1f4d3ab23d93321e8f491d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:20.135397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb366cb-fa27-4f44-8f7b-2cf9f9f1f4cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:20.137848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb366cb-fa27-4f44-8f7b-2cf9f9f1f4cb service placement] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:20.178435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 872/1749] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:20.225291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:20.227884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:20.363314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152590c-aa66-4fff-a878-02d70505f84a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:20.390340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:20.391295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:20.391579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:20.392154 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] PATH_INFO: `/v3/users/9ac573af98574997b4b026ba1ce50058` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:20.393560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac573af98574997b4b026ba1ce50058)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:20.407854 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:20.518534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac573af98574997b4b026ba1ce50058'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.519565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac573af98574997b4b026ba1ce50058'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.538368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac573af98574997b4b026ba1ce50058'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.551832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac573af98574997b4b026ba1ce50058'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.553524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac573af98574997b4b026ba1ce50058'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:20.572874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 878/1750] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:20 2022] DELETE /identity/v3/users/9ac573af98574997b4b026ba1ce50058 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:20.577624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f96691-9310-4d99-97c1-96d31b55e32f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:20.578643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f96691-9310-4d99-97c1-96d31b55e32f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:20.579817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f96691-9310-4d99-97c1-96d31b55e32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:20.623295 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-87f96691-9310-4d99-97c1-96d31b55e32f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:20.630499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-87f96691-9310-4d99-97c1-96d31b55e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:20.666651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:20.668576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:20.681349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:20.792368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 873/1751] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:20.842512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef026f99-0bbd-4c41-beb9-614eb56acb47 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:20.850747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:20.851437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:20.851713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:20.851984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] PATH_INFO: `/v3/projects/098be4758aeb4378afdd49c3e99c59a7` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:20.853113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=098be4758aeb4378afdd49c3e99c59a7)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:20.855113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:20.880653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f96691-9310-4d99-97c1-96d31b55e32f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:20.882555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-87f96691-9310-4d99-97c1-96d31b55e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:20.981098 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098be4758aeb4378afdd49c3e99c59a7', 'request_id': 'req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.005044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f96691-9310-4d99-97c1-96d31b55e32f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:21.012321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098be4758aeb4378afdd49c3e99c59a7', 'request_id': 'req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.021699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098be4758aeb4378afdd49c3e99c59a7', 'request_id': 'req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.043303 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] The token cache is being invalidate because project 098be4758aeb4378afdd49c3e99c59a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:21.045836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.047320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.054348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 879/1752] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:20 2022] DELETE /identity/v3/projects/098be4758aeb4378afdd49c3e99c59a7 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:21.058378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:21.060093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:21.152053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916393a-6832-47a0-8219-0e8582e5a354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:21.152798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916393a-6832-47a0-8219-0e8582e5a354 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:21.153141 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916393a-6832-47a0-8219-0e8582e5a354 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:21.153464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916393a-6832-47a0-8219-0e8582e5a354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:21.155506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0916393a-6832-47a0-8219-0e8582e5a354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:21.208158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ee135-4ac9-42b9-9099-7a6c5e2782b2 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:21.212841 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:21.213485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:21.213755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:21.214020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] PATH_INFO: `/v3/users/07abd7cfc70a49a597d686f49c88d25f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:21.215184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] RBAC: Authorizing `identity:delete_user(user_id=07abd7cfc70a49a597d686f49c88d25f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:21.217127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:21.238861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:21.242649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916393a-6832-47a0-8219-0e8582e5a354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:21.244521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916393a-6832-47a0-8219-0e8582e5a354 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:21.258112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 874/1753] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:21.259632 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513664ab-91b0-4e6f-a18d-fe1988cbf08e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:21.259912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513664ab-91b0-4e6f-a18d-fe1988cbf08e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:21.260178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513664ab-91b0-4e6f-a18d-fe1988cbf08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:21.264035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:21.280544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07abd7cfc70a49a597d686f49c88d25f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.281275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07abd7cfc70a49a597d686f49c88d25f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.301475 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-513664ab-91b0-4e6f-a18d-fe1988cbf08e None None] MFA Rules not processed for user `fec54d8c1ea24cc8bbbeb6fef1fd2028`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:21.301673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07abd7cfc70a49a597d686f49c88d25f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.305922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-513664ab-91b0-4e6f-a18d-fe1988cbf08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:21.319161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07abd7cfc70a49a597d686f49c88d25f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.320829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07abd7cfc70a49a597d686f49c88d25f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.344191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 880/1754] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:21 2022] DELETE /identity/v3/users/07abd7cfc70a49a597d686f49c88d25f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:21.426363 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 875/1755] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:21.430784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:21.433069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:21.557933 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1700d87-96a7-4d21-aa1e-24242f60772c admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:21.564636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b940927-4293-4a79-9d56-fd42ef291996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:21.570972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513664ab-91b0-4e6f-a18d-fe1988cbf08e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:21.571921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b940927-4293-4a79-9d56-fd42ef291996 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:21.572275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b940927-4293-4a79-9d56-fd42ef291996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:21.572551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b940927-4293-4a79-9d56-fd42ef291996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:21.573331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-513664ab-91b0-4e6f-a18d-fe1988cbf08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:21.573453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b940927-4293-4a79-9d56-fd42ef291996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:21.576203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b940927-4293-4a79-9d56-fd42ef291996 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:21.592781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513664ab-91b0-4e6f-a18d-fe1988cbf08e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:21.599474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:21.600224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:21.601288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:21.601288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] PATH_INFO: `/v3/projects/ed35368fcaa741e0b0b769b93d126f17` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:21.602239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed35368fcaa741e0b0b769b93d126f17)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:21.605078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:21.618913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 881/1756] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:21 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:21.623224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b940927-4293-4a79-9d56-fd42ef291996 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:21.685850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed35368fcaa741e0b0b769b93d126f17', 'request_id': 'req-a377a8d1-e761-448f-af77-f591a3bc0acc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.696942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed35368fcaa741e0b0b769b93d126f17', 'request_id': 'req-a377a8d1-e761-448f-af77-f591a3bc0acc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.706467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed35368fcaa741e0b0b769b93d126f17', 'request_id': 'req-a377a8d1-e761-448f-af77-f591a3bc0acc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.718609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] The token cache is being invalidate because project ed35368fcaa741e0b0b769b93d126f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:21.718903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.719581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.721511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 876/1757] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:21 2022] DELETE /identity/v3/projects/ed35368fcaa741e0b0b769b93d126f17 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:21.722373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:21.724290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:21.745247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b940927-4293-4a79-9d56-fd42ef291996 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:21.810880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:21.835167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377a8d1-e761-448f-af77-f591a3bc0acc admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:21.845063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9d909-51e7-4142-8ff7-22b43c77f6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:21.846162 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d909-51e7-4142-8ff7-22b43c77f6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:21.846162 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d909-51e7-4142-8ff7-22b43c77f6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:21.846355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d909-51e7-4142-8ff7-22b43c77f6f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:21.846857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9d909-51e7-4142-8ff7-22b43c77f6f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:21.848853 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9d909-51e7-4142-8ff7-22b43c77f6f2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:21.853095 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:21.853742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:21.854020 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:21.854290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] PATH_INFO: `/v3/projects/042b27b19bf34bdab59f3c1f6ac2c968` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:21.860568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=042b27b19bf34bdab59f3c1f6ac2c968)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:21.862519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:21.881423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:21.909388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 882/1758] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:21 2022] POST /identity/v3/users => generated 299 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:21.911085 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00432756-2a41-448f-a72f-54ba522302b9 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:21.911673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00432756-2a41-448f-a72f-54ba522302b9 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:21.912095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00432756-2a41-448f-a72f-54ba522302b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:21.944654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-00432756-2a41-448f-a72f-54ba522302b9 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:21.946309 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-00432756-2a41-448f-a72f-54ba522302b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:21.957501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '042b27b19bf34bdab59f3c1f6ac2c968', 'request_id': 'req-1835824e-065d-4c55-99b8-2b6813d2f0c3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.966311 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '042b27b19bf34bdab59f3c1f6ac2c968', 'request_id': 'req-1835824e-065d-4c55-99b8-2b6813d2f0c3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.976437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '042b27b19bf34bdab59f3c1f6ac2c968', 'request_id': 'req-1835824e-065d-4c55-99b8-2b6813d2f0c3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.989059 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] The token cache is being invalidate because project 042b27b19bf34bdab59f3c1f6ac2c968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:21.989347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.990002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:21.991604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 877/1759] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:21 2022] DELETE /identity/v3/projects/042b27b19bf34bdab59f3c1f6ac2c968 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:21.992360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:21.993697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:22.027324 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 883/1760] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:22.028323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00432756-2a41-448f-a72f-54ba522302b9 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.029704 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-00432756-2a41-448f-a72f-54ba522302b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:22.042502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00432756-2a41-448f-a72f-54ba522302b9 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.048489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.049174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.049526 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.049867 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] PATH_INFO: `/v3/users/c1da771ca0634d7bbbe1de7e99f7dea8` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.051121 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1da771ca0634d7bbbe1de7e99f7dea8)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.053189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.069714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835824e-065d-4c55-99b8-2b6813d2f0c3 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.076644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734fd86-4ae5-4dd4-a276-0e668b69d888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.077363 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734fd86-4ae5-4dd4-a276-0e668b69d888 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.077768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734fd86-4ae5-4dd4-a276-0e668b69d888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.078470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734fd86-4ae5-4dd4-a276-0e668b69d888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.079696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734fd86-4ae5-4dd4-a276-0e668b69d888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.082587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734fd86-4ae5-4dd4-a276-0e668b69d888 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.099077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 878/1761] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:22.099077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc1eca-e176-4270-bf7c-c26ebcee19a0 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.099077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc1eca-e176-4270-bf7c-c26ebcee19a0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.099077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc1eca-e176-4270-bf7c-c26ebcee19a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.113363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1da771ca0634d7bbbe1de7e99f7dea8'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.114216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1da771ca0634d7bbbe1de7e99f7dea8'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.122793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6bdc1eca-e176-4270-bf7c-c26ebcee19a0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:22.124314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6bdc1eca-e176-4270-bf7c-c26ebcee19a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:22.129626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1da771ca0634d7bbbe1de7e99f7dea8'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.135122 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 879/1762] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:22.135999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc1eca-e176-4270-bf7c-c26ebcee19a0 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.138375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6bdc1eca-e176-4270-bf7c-c26ebcee19a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:22.138779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1da771ca0634d7bbbe1de7e99f7dea8'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.139558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1da771ca0634d7bbbe1de7e99f7dea8'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.155176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc1eca-e176-4270-bf7c-c26ebcee19a0 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.155709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 884/1763] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:22 2022] DELETE /identity/v3/users/c1da771ca0634d7bbbe1de7e99f7dea8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:22.157820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.227840 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:22.234296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6971b4-4c52-4350-a12f-dd6e253a276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.235118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6971b4-4c52-4350-a12f-dd6e253a276e admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.235473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6971b4-4c52-4350-a12f-dd6e253a276e admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.235817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6971b4-4c52-4350-a12f-dd6e253a276e admin admin] PATH_INFO: `/v3/projects/8ac2b0054b6b40a7ae811f72227670d3/users/7f7c84df58964308a4a79228338b2703/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.247399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951fec0-d31b-45e4-9c0f-dd32f4961076 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.256494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ab29a-7d36-48b2-9aa7-2c888f2a2deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.257421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ab29a-7d36-48b2-9aa7-2c888f2a2deb admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.257611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ab29a-7d36-48b2-9aa7-2c888f2a2deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.258129 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ab29a-7d36-48b2-9aa7-2c888f2a2deb admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.258863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ab29a-7d36-48b2-9aa7-2c888f2a2deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:22.259550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ab29a-7d36-48b2-9aa7-2c888f2a2deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.261982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6971b4-4c52-4350-a12f-dd6e253a276e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac2b0054b6b40a7ae811f72227670d3, user_id=7f7c84df58964308a4a79228338b2703, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.262317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ab29a-7d36-48b2-9aa7-2c888f2a2deb admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.266371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6971b4-4c52-4350-a12f-dd6e253a276e admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.283918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 885/1764] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:07:22.288058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 880/1765] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:22 2022] PUT /identity/v3/projects/8ac2b0054b6b40a7ae811f72227670d3/users/7f7c84df58964308a4a79228338b2703/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:22.288433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ab29a-7d36-48b2-9aa7-2c888f2a2deb admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.292803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6971b4-4c52-4350-a12f-dd6e253a276e admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.302236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ab29a-7d36-48b2-9aa7-2c888f2a2deb admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.306799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:22.395619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6971b4-4c52-4350-a12f-dd6e253a276e admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.395938 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44359a5e-1799-4b40-8a27-9cb6f7e2766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.396780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44359a5e-1799-4b40-8a27-9cb6f7e2766b admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.397298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44359a5e-1799-4b40-8a27-9cb6f7e2766b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.397678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44359a5e-1799-4b40-8a27-9cb6f7e2766b admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.398600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44359a5e-1799-4b40-8a27-9cb6f7e2766b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.401383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44359a5e-1799-4b40-8a27-9cb6f7e2766b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.403311 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ca2d9-8657-42d4-ace8-66ed2ed6dfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.403935 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ca2d9-8657-42d4-ace8-66ed2ed6dfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.404206 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ca2d9-8657-42d4-ace8-66ed2ed6dfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.404467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ca2d9-8657-42d4-ace8-66ed2ed6dfc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.405116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ca2d9-8657-42d4-ace8-66ed2ed6dfc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.406928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ca2d9-8657-42d4-ace8-66ed2ed6dfc0 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.423021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 881/1766] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:22.425678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ca2d9-8657-42d4-ace8-66ed2ed6dfc0 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.435666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ca2d9-8657-42d4-ace8-66ed2ed6dfc0 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.436714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 886/1767] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:22 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:22.440858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44359a5e-1799-4b40-8a27-9cb6f7e2766b admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.525986 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44359a5e-1799-4b40-8a27-9cb6f7e2766b admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.531847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bd38f-dfa0-4568-8a10-415e719a83cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.532736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bd38f-dfa0-4568-8a10-415e719a83cd admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.533121 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bd38f-dfa0-4568-8a10-415e719a83cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.533403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bd38f-dfa0-4568-8a10-415e719a83cd admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.534295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bd38f-dfa0-4568-8a10-415e719a83cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.538411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bd38f-dfa0-4568-8a10-415e719a83cd admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.541625 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65b867-9365-4aaf-9304-e9851a6304c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.542518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65b867-9365-4aaf-9304-e9851a6304c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.542975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65b867-9365-4aaf-9304-e9851a6304c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.543898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65b867-9365-4aaf-9304-e9851a6304c4 admin admin] PATH_INFO: `/v3/projects/8ac2b0054b6b40a7ae811f72227670d3/users/7f7c84df58964308a4a79228338b2703/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.547401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65b867-9365-4aaf-9304-e9851a6304c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac2b0054b6b40a7ae811f72227670d3, user_id=7f7c84df58964308a4a79228338b2703, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.552776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65b867-9365-4aaf-9304-e9851a6304c4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.569083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 882/1768] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:22 2022] PUT /identity/v3/projects/8ac2b0054b6b40a7ae811f72227670d3/users/7f7c84df58964308a4a79228338b2703/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:22.576456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65b867-9365-4aaf-9304-e9851a6304c4 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.577852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 887/1769] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:22 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:22.662363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bd38f-dfa0-4568-8a10-415e719a83cd admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.664314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-768bd38f-dfa0-4568-8a10-415e719a83cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:22.682628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bd38f-dfa0-4568-8a10-415e719a83cd admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.711017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65b867-9365-4aaf-9304-e9851a6304c4 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.716699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e784d-3e71-4908-8985-a351126d32b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.717327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e784d-3e71-4908-8985-a351126d32b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.717662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e784d-3e71-4908-8985-a351126d32b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.717939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e784d-3e71-4908-8985-a351126d32b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.719192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e784d-3e71-4908-8985-a351126d32b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.721507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e784d-3e71-4908-8985-a351126d32b2 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.741238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 883/1770] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:22.745130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e784d-3e71-4908-8985-a351126d32b2 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.756481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:22.764662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e784d-3e71-4908-8985-a351126d32b2 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.766138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.767639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.767922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.768199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] PATH_INFO: `/v3/projects/5431eb1c3cab4b50a609a2d6ea3421c6` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.769457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5431eb1c3cab4b50a609a2d6ea3421c6)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.771464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.772637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf61ab4-c00e-4ae1-bcd0-3a83ac27456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.773660 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf61ab4-c00e-4ae1-bcd0-3a83ac27456a admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.773999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf61ab4-c00e-4ae1-bcd0-3a83ac27456a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.774385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf61ab4-c00e-4ae1-bcd0-3a83ac27456a admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.775462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf61ab4-c00e-4ae1-bcd0-3a83ac27456a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.777214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf61ab4-c00e-4ae1-bcd0-3a83ac27456a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.795223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 884/1771] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:22.796446 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf61ab4-c00e-4ae1-bcd0-3a83ac27456a admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:22.807724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf61ab4-c00e-4ae1-bcd0-3a83ac27456a admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:22.813692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0055f-b34a-467f-bb80-807028d4d477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:22.814377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0055f-b34a-467f-bb80-807028d4d477 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.814668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0055f-b34a-467f-bb80-807028d4d477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.814937 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0055f-b34a-467f-bb80-807028d4d477 admin admin] PATH_INFO: `/v3/projects/a0cd6939d0e9465ca89cd23245ebc257/users/1adbdd5f59de4341ac3d870ce21b63b7/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.841015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0055f-b34a-467f-bb80-807028d4d477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0cd6939d0e9465ca89cd23245ebc257, user_id=1adbdd5f59de4341ac3d870ce21b63b7, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:22.846005 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0055f-b34a-467f-bb80-807028d4d477 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:22.847912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5431eb1c3cab4b50a609a2d6ea3421c6', 'request_id': 'req-fbc595ae-f641-43c6-bf62-39df115b568a'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.859820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5431eb1c3cab4b50a609a2d6ea3421c6', 'request_id': 'req-fbc595ae-f641-43c6-bf62-39df115b568a'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.860228 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:22.869208 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5431eb1c3cab4b50a609a2d6ea3421c6', 'request_id': 'req-fbc595ae-f641-43c6-bf62-39df115b568a'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.881675 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 885/1772] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:22 2022] PUT /identity/v3/projects/a0cd6939d0e9465ca89cd23245ebc257/users/1adbdd5f59de4341ac3d870ce21b63b7/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:22.883629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] The token cache is being invalidate because project 5431eb1c3cab4b50a609a2d6ea3421c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:22.883751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b903f-d213-4eac-b0ca-1e5d1ece048f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.884036 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.884141 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b903f-d213-4eac-b0ca-1e5d1ece048f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.884326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b903f-d213-4eac-b0ca-1e5d1ece048f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.884818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fbc595ae-f641-43c6-bf62-39df115b568a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:22.886855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 888/1773] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:22 2022] DELETE /identity/v3/projects/5431eb1c3cab4b50a609a2d6ea3421c6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:22.891503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c07cd-035b-4dd0-83b8-8c9e32b0efe2 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:22.892201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c07cd-035b-4dd0-83b8-8c9e32b0efe2 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:22.892201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c07cd-035b-4dd0-83b8-8c9e32b0efe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:22.921178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-316c07cd-035b-4dd0-83b8-8c9e32b0efe2 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:22.922799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-316c07cd-035b-4dd0-83b8-8c9e32b0efe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:22.933900 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-404b903f-d213-4eac-b0ca-1e5d1ece048f None None] MFA Rules not processed for user `1adbdd5f59de4341ac3d870ce21b63b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:22.936135 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-404b903f-d213-4eac-b0ca-1e5d1ece048f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:23.039769 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 886/1774] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:23.054860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b903f-d213-4eac-b0ca-1e5d1ece048f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:23.056441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-404b903f-d213-4eac-b0ca-1e5d1ece048f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:23.056626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 889/1775] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:23.065616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c9094-28ec-4133-8c0d-66f0d262eec3 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.066103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c9094-28ec-4133-8c0d-66f0d262eec3 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.066957 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c9094-28ec-4133-8c0d-66f0d262eec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.090637 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-d63c9094-28ec-4133-8c0d-66f0d262eec3 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:23.092297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d63c9094-28ec-4133-8c0d-66f0d262eec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:23.106420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 890/1776] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:23.111234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c9094-28ec-4133-8c0d-66f0d262eec3 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:23.130902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c9094-28ec-4133-8c0d-66f0d262eec3 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:23.144222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d195-b43a-47de-b459-b33119b186c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:23.152287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d195-b43a-47de-b459-b33119b186c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.152287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d195-b43a-47de-b459-b33119b186c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.152287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d195-b43a-47de-b459-b33119b186c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.152287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071d195-b43a-47de-b459-b33119b186c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:23.152287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071d195-b43a-47de-b459-b33119b186c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:23.154212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071d195-b43a-47de-b459-b33119b186c1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:23.171903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 891/1777] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:07:23.176160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d195-b43a-47de-b459-b33119b186c1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:23.187123 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b903f-d213-4eac-b0ca-1e5d1ece048f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:23.193112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54082135-7971-4dc2-9b85-57626a2e21c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:23.194963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54082135-7971-4dc2-9b85-57626a2e21c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.195345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54082135-7971-4dc2-9b85-57626a2e21c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.195717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54082135-7971-4dc2-9b85-57626a2e21c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.196588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d195-b43a-47de-b459-b33119b186c1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:23.208428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54082135-7971-4dc2-9b85-57626a2e21c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:23.209278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7031d-a27e-44ee-8496-45b9006c791b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:23.210451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7031d-a27e-44ee-8496-45b9006c791b admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.210573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54082135-7971-4dc2-9b85-57626a2e21c0 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:23.210788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7031d-a27e-44ee-8496-45b9006c791b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.211089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7031d-a27e-44ee-8496-45b9006c791b admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.211729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7031d-a27e-44ee-8496-45b9006c791b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:23.213709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7031d-a27e-44ee-8496-45b9006c791b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:23.227945 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 887/1778] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:07:23.269910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54082135-7971-4dc2-9b85-57626a2e21c0 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:23.272089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-54082135-7971-4dc2-9b85-57626a2e21c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:23.291265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 892/1779] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:23 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:23.292929 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7031d-a27e-44ee-8496-45b9006c791b admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:23.300195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54082135-7971-4dc2-9b85-57626a2e21c0 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:23.354684 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:23.388882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7031d-a27e-44ee-8496-45b9006c791b admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:23.396155 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8176f86-21b2-4e67-b455-1682850001ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:23.396916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8176f86-21b2-4e67-b455-1682850001ac admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.397303 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8176f86-21b2-4e67-b455-1682850001ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.397660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8176f86-21b2-4e67-b455-1682850001ac admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.398482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8176f86-21b2-4e67-b455-1682850001ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:23.400946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8176f86-21b2-4e67-b455-1682850001ac admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:23.419506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:23.420326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.420747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.439567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] PATH_INFO: `/v3/users/1adbdd5f59de4341ac3d870ce21b63b7` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.446164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1adbdd5f59de4341ac3d870ce21b63b7)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:23.448632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 893/1780] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:23 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:23.450895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:23.461159 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8176f86-21b2-4e67-b455-1682850001ac admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:23.493473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8176f86-21b2-4e67-b455-1682850001ac admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:23.501445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd5d66-57db-4f1b-9f03-b97d3dc3644d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:23.502295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd5d66-57db-4f1b-9f03-b97d3dc3644d admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.516143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd5d66-57db-4f1b-9f03-b97d3dc3644d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.516563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd5d66-57db-4f1b-9f03-b97d3dc3644d admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.519005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd5d66-57db-4f1b-9f03-b97d3dc3644d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:23.522559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd5d66-57db-4f1b-9f03-b97d3dc3644d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:23.549793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 894/1781] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:23.553348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd5d66-57db-4f1b-9f03-b97d3dc3644d admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:23.575830 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd5d66-57db-4f1b-9f03-b97d3dc3644d admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:23.580402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:23.589941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155f09f-73bb-4337-a570-a5c719f71d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:23.590880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155f09f-73bb-4337-a570-a5c719f71d51 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.593066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155f09f-73bb-4337-a570-a5c719f71d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.593852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155f09f-73bb-4337-a570-a5c719f71d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.594325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1adbdd5f59de4341ac3d870ce21b63b7'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:23.599023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1adbdd5f59de4341ac3d870ce21b63b7'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:23.601009 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155f09f-73bb-4337-a570-a5c719f71d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:23.607874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155f09f-73bb-4337-a570-a5c719f71d51 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:23.663343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1adbdd5f59de4341ac3d870ce21b63b7'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:23.674568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 895/1782] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:23.677756 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155f09f-73bb-4337-a570-a5c719f71d51 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:23.681253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8155f09f-73bb-4337-a570-a5c719f71d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:23.690741 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1adbdd5f59de4341ac3d870ce21b63b7'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:23.705732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16dba022-a7b3-44db-8f17-3acef3554c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1adbdd5f59de4341ac3d870ce21b63b7'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:23.711368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155f09f-73bb-4337-a570-a5c719f71d51 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:23.715196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 888/1783] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:23 2022] DELETE /identity/v3/users/1adbdd5f59de4341ac3d870ce21b63b7 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:23.722076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cf39e-7d99-49d9-89bc-86c488541cd0 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.722454 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cf39e-7d99-49d9-89bc-86c488541cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.722781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cf39e-7d99-49d9-89bc-86c488541cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.775497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-5a9cf39e-7d99-49d9-89bc-86c488541cd0 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:23.777288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5a9cf39e-7d99-49d9-89bc-86c488541cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:23.811383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba5e11-5f4a-4d95-b6bb-2961be88bc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:23.812750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba5e11-5f4a-4d95-b6bb-2961be88bc2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.813170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba5e11-5f4a-4d95-b6bb-2961be88bc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.813470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba5e11-5f4a-4d95-b6bb-2961be88bc2b admin admin] PATH_INFO: `/v3/projects/3cda39058bee49879e071275e52e4f31/users/2d91c33ff20f4b8ab4d3ac9db0ab815c/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.837313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba5e11-5f4a-4d95-b6bb-2961be88bc2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cda39058bee49879e071275e52e4f31, user_id=2d91c33ff20f4b8ab4d3ac9db0ab815c, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:23.841245 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba5e11-5f4a-4d95-b6bb-2961be88bc2b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:23.864379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 896/1784] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:23 2022] PUT /identity/v3/projects/3cda39058bee49879e071275e52e4f31/users/2d91c33ff20f4b8ab4d3ac9db0ab815c/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:23.875461 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a9133-4ceb-4071-b738-c266b76c3036 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:23.875841 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a9133-4ceb-4071-b738-c266b76c3036 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:23.876020 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a9133-4ceb-4071-b738-c266b76c3036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:23.895341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 889/1785] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:23.906680 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cf39e-7d99-49d9-89bc-86c488541cd0 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:23.908743 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5a9cf39e-7d99-49d9-89bc-86c488541cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:23.911148 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-051a9133-4ceb-4071-b738-c266b76c3036 None None] MFA Rules not processed for user `2d91c33ff20f4b8ab4d3ac9db0ab815c`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:23.912355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-051a9133-4ceb-4071-b738-c266b76c3036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:23.994832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 897/1786] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:23.999323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a9133-4ceb-4071-b738-c266b76c3036 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:24.008515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cf39e-7d99-49d9-89bc-86c488541cd0 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:24.036808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:24.077934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a9133-4ceb-4071-b738-c266b76c3036 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:24.090533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9d060-c4e4-4088-84d9-f40531ecf062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:24.091755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d060-c4e4-4088-84d9-f40531ecf062 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.092864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d060-c4e4-4088-84d9-f40531ecf062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.093289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d060-c4e4-4088-84d9-f40531ecf062 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.094468 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9d060-c4e4-4088-84d9-f40531ecf062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:24.096887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9d060-c4e4-4088-84d9-f40531ecf062 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:24.098813 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222b89-589e-4736-a22d-e8f3f341f4ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:24.099716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222b89-589e-4736-a22d-e8f3f341f4ae service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.100096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222b89-589e-4736-a22d-e8f3f341f4ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.100470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222b89-589e-4736-a22d-e8f3f341f4ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.120414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222b89-589e-4736-a22d-e8f3f341f4ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:24.122471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222b89-589e-4736-a22d-e8f3f341f4ae service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:24.136357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 890/1787] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:24.163816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222b89-589e-4736-a22d-e8f3f341f4ae service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:24.172657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 898/1788] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:23 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:24.184194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222b89-589e-4736-a22d-e8f3f341f4ae service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:24.196099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:24.316636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3687f-3684-48fb-93d2-1a8294550b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:24.320376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3687f-3684-48fb-93d2-1a8294550b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.321080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3687f-3684-48fb-93d2-1a8294550b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.321399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3687f-3684-48fb-93d2-1a8294550b5e admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.322309 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3687f-3684-48fb-93d2-1a8294550b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:24.325021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3687f-3684-48fb-93d2-1a8294550b5e admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:24.362063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9d060-c4e4-4088-84d9-f40531ecf062 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:24.364000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 891/1789] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:24 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:24.378234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9d060-c4e4-4088-84d9-f40531ecf062 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:24.387213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e86957-78ff-4321-af52-ce1732c44123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:24.388033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e86957-78ff-4321-af52-ce1732c44123 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.388407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e86957-78ff-4321-af52-ce1732c44123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.388777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e86957-78ff-4321-af52-ce1732c44123 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.389537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e86957-78ff-4321-af52-ce1732c44123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:24.391366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e86957-78ff-4321-af52-ce1732c44123 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:24.410222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 899/1790] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:24.415936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3687f-3684-48fb-93d2-1a8294550b5e admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:24.431396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3687f-3684-48fb-93d2-1a8294550b5e admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:24.438117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777640db-421e-47bb-b6c3-332341247cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:24.438939 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777640db-421e-47bb-b6c3-332341247cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.439349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777640db-421e-47bb-b6c3-332341247cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.439719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777640db-421e-47bb-b6c3-332341247cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.440533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777640db-421e-47bb-b6c3-332341247cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:24.442318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777640db-421e-47bb-b6c3-332341247cb3 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:24.462113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 892/1791] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:24.465320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e86957-78ff-4321-af52-ce1732c44123 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:24.484852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e86957-78ff-4321-af52-ce1732c44123 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:24.490721 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b2d76-8922-4035-81de-be4010393e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:24.492413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2d76-8922-4035-81de-be4010393e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.492894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2d76-8922-4035-81de-be4010393e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.493281 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2d76-8922-4035-81de-be4010393e91 admin admin] PATH_INFO: `/v3/projects/cf601d6e73984fb3ae2d6cdd19ee8f5c/users/edea4cdd71c840c08a6bb65d6cd837bf/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.515363 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b2d76-8922-4035-81de-be4010393e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf601d6e73984fb3ae2d6cdd19ee8f5c, user_id=edea4cdd71c840c08a6bb65d6cd837bf, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:24.519173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b2d76-8922-4035-81de-be4010393e91 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:24.543402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 900/1792] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:24 2022] PUT /identity/v3/projects/cf601d6e73984fb3ae2d6cdd19ee8f5c/users/edea4cdd71c840c08a6bb65d6cd837bf/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:24.552642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8d1f-e6e9-40bc-9ce5-5a57084d5bd3 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.553626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8d1f-e6e9-40bc-9ce5-5a57084d5bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.554000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8d1f-e6e9-40bc-9ce5-5a57084d5bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.571364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b2d76-8922-4035-81de-be4010393e91 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:24.574914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-143b2d76-8922-4035-81de-be4010393e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:24.595418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-eeab8d1f-e6e9-40bc-9ce5-5a57084d5bd3 None None] MFA Rules not processed for user `edea4cdd71c840c08a6bb65d6cd837bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:24.596759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-eeab8d1f-e6e9-40bc-9ce5-5a57084d5bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:24.737118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b2d76-8922-4035-81de-be4010393e91 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:24.742699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:24.744490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.744490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.744490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] PATH_INFO: `/v3/projects/a0cd6939d0e9465ca89cd23245ebc257` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.745873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0cd6939d0e9465ca89cd23245ebc257)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:24.748357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:24.786365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 893/1793] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:24.792085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab8d1f-e6e9-40bc-9ce5-5a57084d5bd3 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:24.815413 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab8d1f-e6e9-40bc-9ce5-5a57084d5bd3 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:24.826883 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24353aa0-108d-4dc9-9df8-73b9587b9fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:24.827673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24353aa0-108d-4dc9-9df8-73b9587b9fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.829204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24353aa0-108d-4dc9-9df8-73b9587b9fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.829584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24353aa0-108d-4dc9-9df8-73b9587b9fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.830342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24353aa0-108d-4dc9-9df8-73b9587b9fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:24.836631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0cd6939d0e9465ca89cd23245ebc257', 'request_id': 'req-08dd8967-0698-4293-8b90-024094ed8268'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:24.837302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24353aa0-108d-4dc9-9df8-73b9587b9fe6 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:24.856554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0cd6939d0e9465ca89cd23245ebc257', 'request_id': 'req-08dd8967-0698-4293-8b90-024094ed8268'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:24.876496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0cd6939d0e9465ca89cd23245ebc257', 'request_id': 'req-08dd8967-0698-4293-8b90-024094ed8268'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:24.900776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 894/1794] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:24 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:24.909263 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24353aa0-108d-4dc9-9df8-73b9587b9fe6 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:24.913631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] The token cache is being invalidate because project a0cd6939d0e9465ca89cd23245ebc257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:24.914815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:24.916397 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-08dd8967-0698-4293-8b90-024094ed8268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:24.925596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 901/1795] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:24 2022] DELETE /identity/v3/projects/a0cd6939d0e9465ca89cd23245ebc257 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:24.929947 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcedbf-92e8-42b3-99c2-e46b25130e24 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:24.930224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcedbf-92e8-42b3-99c2-e46b25130e24 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:24.930484 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcedbf-92e8-42b3-99c2-e46b25130e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:24.961988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:24.970414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-efbcedbf-92e8-42b3-99c2-e46b25130e24 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:24.977555 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-efbcedbf-92e8-42b3-99c2-e46b25130e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:25.060716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:25.065459 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24353aa0-108d-4dc9-9df8-73b9587b9fe6 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.082033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c778d-5b57-4079-b72a-ac4a8b2d5137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.083755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c778d-5b57-4079-b72a-ac4a8b2d5137 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.084395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c778d-5b57-4079-b72a-ac4a8b2d5137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.084904 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c778d-5b57-4079-b72a-ac4a8b2d5137 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.085651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c778d-5b57-4079-b72a-ac4a8b2d5137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.088132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c778d-5b57-4079-b72a-ac4a8b2d5137 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.126783 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 902/1796] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:25.129472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65ca69-ba29-4ae7-a02d-290d197a6b34 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.129816 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65ca69-ba29-4ae7-a02d-290d197a6b34 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.130180 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65ca69-ba29-4ae7-a02d-290d197a6b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.146441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 895/1797] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:24 2022] POST /identity/v3/users => generated 299 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:25.150814 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c778d-5b57-4079-b72a-ac4a8b2d5137 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.154728 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f42c778d-5b57-4079-b72a-ac4a8b2d5137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:25.160469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8e65ca69-ba29-4ae7-a02d-290d197a6b34 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:25.160947 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8e65ca69-ba29-4ae7-a02d-290d197a6b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:25.176038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c778d-5b57-4079-b72a-ac4a8b2d5137 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.179648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 903/1798] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:25.184050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80d142-9b3d-4bb4-966a-557d6f32e7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.184343 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65ca69-ba29-4ae7-a02d-290d197a6b34 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.184938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80d142-9b3d-4bb4-966a-557d6f32e7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.185323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80d142-9b3d-4bb4-966a-557d6f32e7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.185700 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80d142-9b3d-4bb4-966a-557d6f32e7f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.186714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80d142-9b3d-4bb4-966a-557d6f32e7f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.189160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80d142-9b3d-4bb4-966a-557d6f32e7f1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.196287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65ca69-ba29-4ae7-a02d-290d197a6b34 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.202758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010ee6e-e47a-476c-8d93-f7de32ecb647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.204117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010ee6e-e47a-476c-8d93-f7de32ecb647 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.204569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010ee6e-e47a-476c-8d93-f7de32ecb647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.204985 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010ee6e-e47a-476c-8d93-f7de32ecb647 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.205788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010ee6e-e47a-476c-8d93-f7de32ecb647 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:25.206239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010ee6e-e47a-476c-8d93-f7de32ecb647 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.208083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010ee6e-e47a-476c-8d93-f7de32ecb647 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.221697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 896/1799] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:25.226142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80d142-9b3d-4bb4-966a-557d6f32e7f1 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.237161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 904/1800] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:07:25.238623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010ee6e-e47a-476c-8d93-f7de32ecb647 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.251378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80d142-9b3d-4bb4-966a-557d6f32e7f1 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.253219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010ee6e-e47a-476c-8d93-f7de32ecb647 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.257832 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f69aa-12d2-43b1-804d-0b36898204bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.258731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f69aa-12d2-43b1-804d-0b36898204bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.259241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f69aa-12d2-43b1-804d-0b36898204bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.259757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f69aa-12d2-43b1-804d-0b36898204bc admin admin] PATH_INFO: `/v3/projects/df17840a56d342df8cc3703be5c6dbcc/users/52c06b565d194c8085242f064e65b99b/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.260224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e261c5-8897-4aa3-9384-1dbaf12432dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.261044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e261c5-8897-4aa3-9384-1dbaf12432dc admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.261946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e261c5-8897-4aa3-9384-1dbaf12432dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.262327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e261c5-8897-4aa3-9384-1dbaf12432dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.264095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e261c5-8897-4aa3-9384-1dbaf12432dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.270695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e261c5-8897-4aa3-9384-1dbaf12432dc admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.301171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f69aa-12d2-43b1-804d-0b36898204bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=df17840a56d342df8cc3703be5c6dbcc, user_id=52c06b565d194c8085242f064e65b99b, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.310950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f69aa-12d2-43b1-804d-0b36898204bc admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.323944 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 905/1801] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:25 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:25.330264 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e261c5-8897-4aa3-9384-1dbaf12432dc admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.345124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 897/1802] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:25 2022] PUT /identity/v3/projects/df17840a56d342df8cc3703be5c6dbcc/users/52c06b565d194c8085242f064e65b99b/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:25.349709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f69aa-12d2-43b1-804d-0b36898204bc admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.433768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e261c5-8897-4aa3-9384-1dbaf12432dc admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.460986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f69aa-12d2-43b1-804d-0b36898204bc admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.469332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524825d2-d46f-42fd-8c77-2c616886fd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.470159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524825d2-d46f-42fd-8c77-2c616886fd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.470503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524825d2-d46f-42fd-8c77-2c616886fd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.470863 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524825d2-d46f-42fd-8c77-2c616886fd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.471691 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524825d2-d46f-42fd-8c77-2c616886fd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.473553 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524825d2-d46f-42fd-8c77-2c616886fd4c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.487915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 898/1803] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:25.492768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524825d2-d46f-42fd-8c77-2c616886fd4c admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.508783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524825d2-d46f-42fd-8c77-2c616886fd4c admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.515259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acce25-7617-4cfd-82d7-5c574b624b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.516299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acce25-7617-4cfd-82d7-5c574b624b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.516572 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acce25-7617-4cfd-82d7-5c574b624b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.516848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acce25-7617-4cfd-82d7-5c574b624b7d admin admin] PATH_INFO: `/v3/projects/df17840a56d342df8cc3703be5c6dbcc/users/52c06b565d194c8085242f064e65b99b/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.518911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acce25-7617-4cfd-82d7-5c574b624b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df17840a56d342df8cc3703be5c6dbcc, user_id=52c06b565d194c8085242f064e65b99b, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.521922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acce25-7617-4cfd-82d7-5c574b624b7d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.534808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 899/1804] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:25 2022] PUT /identity/v3/projects/df17840a56d342df8cc3703be5c6dbcc/users/52c06b565d194c8085242f064e65b99b/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:25.550296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f6b6a-71c4-4717-b3bd-6d73baff7ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.551407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e2716-9f92-4c98-b685-c5bdb82f945d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.551532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f6b6a-71c4-4717-b3bd-6d73baff7ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.552169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e2716-9f92-4c98-b685-c5bdb82f945d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.552169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e2716-9f92-4c98-b685-c5bdb82f945d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.553777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f6b6a-71c4-4717-b3bd-6d73baff7ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.554657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f6b6a-71c4-4717-b3bd-6d73baff7ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.555692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f6b6a-71c4-4717-b3bd-6d73baff7ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.564436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f6b6a-71c4-4717-b3bd-6d73baff7ae5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.592772 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6f1e2716-9f92-4c98-b685-c5bdb82f945d None None] MFA Rules not processed for user `52c06b565d194c8085242f064e65b99b`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:25.595011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6f1e2716-9f92-4c98-b685-c5bdb82f945d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:25.607259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 906/1805] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:25 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:25.612933 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f6b6a-71c4-4717-b3bd-6d73baff7ae5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.621457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:25.637583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f6b6a-71c4-4717-b3bd-6d73baff7ae5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.763414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 900/1806] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:25.790902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e2716-9f92-4c98-b685-c5bdb82f945d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.799208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6f1e2716-9f92-4c98-b685-c5bdb82f945d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:25.827914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0f519-d94d-4dfd-a848-9497f4a5a2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.829381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0f519-d94d-4dfd-a848-9497f4a5a2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.829880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0f519-d94d-4dfd-a848-9497f4a5a2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.830360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0f519-d94d-4dfd-a848-9497f4a5a2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.842246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0f519-d94d-4dfd-a848-9497f4a5a2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.845506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0f519-d94d-4dfd-a848-9497f4a5a2c1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.864111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:25.864976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 907/1807] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:25.889808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0f519-d94d-4dfd-a848-9497f4a5a2c1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:25.919501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0f519-d94d-4dfd-a848-9497f4a5a2c1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:25.941644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bce18-9e89-47b6-bf71-53a64dd7cc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:25.942392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bce18-9e89-47b6-bf71-53a64dd7cc36 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:25.943154 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bce18-9e89-47b6-bf71-53a64dd7cc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:25.943501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bce18-9e89-47b6-bf71-53a64dd7cc36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:25.944239 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bce18-9e89-47b6-bf71-53a64dd7cc36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:25.946317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bce18-9e89-47b6-bf71-53a64dd7cc36 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:25.976460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 908/1808] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:25.977420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bce18-9e89-47b6-bf71-53a64dd7cc36 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:26.004466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e2716-9f92-4c98-b685-c5bdb82f945d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:26.031307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:26.032207 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:26.032558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:26.032903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:26.037251 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:26.050296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bce18-9e89-47b6-bf71-53a64dd7cc36 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:26.063657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55345-c66c-4d66-8291-8011e693fb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:26.064396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55345-c66c-4d66-8291-8011e693fb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:26.064853 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55345-c66c-4d66-8291-8011e693fb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:26.065001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55345-c66c-4d66-8291-8011e693fb3e admin admin] PATH_INFO: `/v3/projects/8360bdaafd854d6b8073e78af1328334/users/755a77b791484f48bf5cfdb62b3ad692/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:26.150581 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe55345-c66c-4d66-8291-8011e693fb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8360bdaafd854d6b8073e78af1328334, user_id=755a77b791484f48bf5cfdb62b3ad692, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:26.155345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe55345-c66c-4d66-8291-8011e693fb3e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:26.178398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 909/1809] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:25 2022] PUT /identity/v3/projects/8360bdaafd854d6b8073e78af1328334/users/755a77b791484f48bf5cfdb62b3ad692/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:26.180088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:26.181788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:26.191652 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5040948-7cca-446c-811c-dabaf5afdd13 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:26.192004 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5040948-7cca-446c-811c-dabaf5afdd13 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:26.192392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5040948-7cca-446c-811c-dabaf5afdd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:26.202344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 901/1810] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:07:26.246785 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f5040948-7cca-446c-811c-dabaf5afdd13 None None] MFA Rules not processed for user `755a77b791484f48bf5cfdb62b3ad692`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:26.248126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f5040948-7cca-446c-811c-dabaf5afdd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:26.377465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 910/1811] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:26.388693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:26.392249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:26.412708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fd9b4-762f-4900-b736-686363a58d64 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:26.564352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a8f98-2737-48c3-a3a8-0b8da32b3cbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:26.565587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a8f98-2737-48c3-a3a8-0b8da32b3cbb service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:26.566041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a8f98-2737-48c3-a3a8-0b8da32b3cbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:26.566609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a8f98-2737-48c3-a3a8-0b8da32b3cbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:26.578476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a8f98-2737-48c3-a3a8-0b8da32b3cbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:26.580105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a8f98-2737-48c3-a3a8-0b8da32b3cbb service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:26.601009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 902/1812] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:28.264460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5040948-7cca-446c-811c-dabaf5afdd13 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:28.285493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5040948-7cca-446c-811c-dabaf5afdd13 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:28.295401 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb129535-4b49-44b7-a4c5-06b6304ffbeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:28.296197 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb129535-4b49-44b7-a4c5-06b6304ffbeb service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:28.297234 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb129535-4b49-44b7-a4c5-06b6304ffbeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:28.297653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb129535-4b49-44b7-a4c5-06b6304ffbeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:28.302655 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cb129535-4b49-44b7-a4c5-06b6304ffbeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:28.446748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:28.464203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb129535-4b49-44b7-a4c5-06b6304ffbeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:28.466861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb129535-4b49-44b7-a4c5-06b6304ffbeb service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:28.513200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 911/1813] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:29.698815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a8f98-2737-48c3-a3a8-0b8da32b3cbb service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:29.711768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a8f98-2737-48c3-a3a8-0b8da32b3cbb service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:29.745376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382b24f-906e-4fac-8a59-b47fae464ce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:29.746018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382b24f-906e-4fac-8a59-b47fae464ce6 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:29.746302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382b24f-906e-4fac-8a59-b47fae464ce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:29.747333 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382b24f-906e-4fac-8a59-b47fae464ce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:29.839284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382b24f-906e-4fac-8a59-b47fae464ce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:29.840734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382b24f-906e-4fac-8a59-b47fae464ce6 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:29.855124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 903/1814] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:30.435582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d353a3f-b746-4ed8-8efa-9edf763966bd None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:30.436769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d353a3f-b746-4ed8-8efa-9edf763966bd None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:30.437757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d353a3f-b746-4ed8-8efa-9edf763966bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:30.479690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8d353a3f-b746-4ed8-8efa-9edf763966bd None None] MFA Rules not processed for user `7f7c84df58964308a4a79228338b2703`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:30.480934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8d353a3f-b746-4ed8-8efa-9edf763966bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:30.579145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 912/1815] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:30.591894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382b24f-906e-4fac-8a59-b47fae464ce6 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:30.593451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3382b24f-906e-4fac-8a59-b47fae464ce6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:30.614086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:30.663032 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382b24f-906e-4fac-8a59-b47fae464ce6 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:30.666538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa63b7d-e3c5-4968-8014-cb7a0d6e471d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8ec16bd9254b23bfd2b60551af9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:30.667220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa63b7d-e3c5-4968-8014-cb7a0d6e471d service glance] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:30.667493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa63b7d-e3c5-4968-8014-cb7a0d6e471d service glance] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:30.667760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa63b7d-e3c5-4968-8014-cb7a0d6e471d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:30.669488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-bfa63b7d-e3c5-4968-8014-cb7a0d6e471d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:30.744443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa63b7d-e3c5-4968-8014-cb7a0d6e471d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:30.746889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa63b7d-e3c5-4968-8014-cb7a0d6e471d service glance] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:30.758176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 904/1816] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:30.789795 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d353a3f-b746-4ed8-8efa-9edf763966bd None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:30.901371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d353a3f-b746-4ed8-8efa-9edf763966bd None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:30.910729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:30.912576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:30.912915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:30.913127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] PATH_INFO: `/v3/users/755a77b791484f48bf5cfdb62b3ad692` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:30.915047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755a77b791484f48bf5cfdb62b3ad692)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:30.917218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:30.963565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df008156-d69f-4f07-9dab-5024a923a345 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:30.963857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df008156-d69f-4f07-9dab-5024a923a345 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:30.964135 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df008156-d69f-4f07-9dab-5024a923a345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:30.987501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755a77b791484f48bf5cfdb62b3ad692'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:30.988670 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755a77b791484f48bf5cfdb62b3ad692'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:30.990335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-df008156-d69f-4f07-9dab-5024a923a345 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:30.991773 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-df008156-d69f-4f07-9dab-5024a923a345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:31.008180 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755a77b791484f48bf5cfdb62b3ad692'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:31.035672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755a77b791484f48bf5cfdb62b3ad692'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:31.036693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-638ea19c-b3d6-448f-b6f7-ed90dfb2e8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755a77b791484f48bf5cfdb62b3ad692'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:31.041359 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:31.072110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 913/1817] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:30 2022] DELETE /identity/v3/users/755a77b791484f48bf5cfdb62b3ad692 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:31.073702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4862650-cd13-4744-bd77-621d862abb6e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:31.074000 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4862650-cd13-4744-bd77-621d862abb6e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:31.074277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4862650-cd13-4744-bd77-621d862abb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:31.077215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 905/1818] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:31.097930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-b4862650-cd13-4744-bd77-621d862abb6e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:31.099365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b4862650-cd13-4744-bd77-621d862abb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:31.146873 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df008156-d69f-4f07-9dab-5024a923a345 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:31.148618 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-df008156-d69f-4f07-9dab-5024a923a345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:31.250601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 914/1819] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:31.264757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4862650-cd13-4744-bd77-621d862abb6e None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:31.265530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df008156-d69f-4f07-9dab-5024a923a345 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:31.267699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b4862650-cd13-4744-bd77-621d862abb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:31.276136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:31.343669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59bfaf-b564-4306-9a2b-73058862162d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:31.344568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59bfaf-b564-4306-9a2b-73058862162d service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:31.345093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59bfaf-b564-4306-9a2b-73058862162d service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:31.345666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59bfaf-b564-4306-9a2b-73058862162d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:31.362428 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4862650-cd13-4744-bd77-621d862abb6e None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:31.370552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e937cdb-6af9-4404-8525-ac6b1df9a5f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:31.371398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e937cdb-6af9-4404-8525-ac6b1df9a5f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:31.371749 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e937cdb-6af9-4404-8525-ac6b1df9a5f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:31.372289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e937cdb-6af9-4404-8525-ac6b1df9a5f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:31.388996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e937cdb-6af9-4404-8525-ac6b1df9a5f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:31.391267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e937cdb-6af9-4404-8525-ac6b1df9a5f3 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:31.405394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 915/1820] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:07:31.474332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59bfaf-b564-4306-9a2b-73058862162d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:31.476464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59bfaf-b564-4306-9a2b-73058862162d service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:31.491092 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 906/1821] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:07:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 28 19:07:32.487985 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e937cdb-6af9-4404-8525-ac6b1df9a5f3 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:32.490572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4e937cdb-6af9-4404-8525-ac6b1df9a5f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:32.508012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e937cdb-6af9-4404-8525-ac6b1df9a5f3 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:32.536155 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:32.541084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:32.541807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:32.541807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] PATH_INFO: `/v3/projects/8360bdaafd854d6b8073e78af1328334` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:32.543657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8360bdaafd854d6b8073e78af1328334)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:32.548023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:32.664760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8360bdaafd854d6b8073e78af1328334', 'request_id': 'req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:32.690065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8360bdaafd854d6b8073e78af1328334', 'request_id': 'req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:32.707288 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8360bdaafd854d6b8073e78af1328334', 'request_id': 'req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:32.736722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] The token cache is being invalidate because project 8360bdaafd854d6b8073e78af1328334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:32.737325 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:32.739084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6ab80a2d-995d-43cd-ad14-9b5f37f44db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:32.746356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 916/1822] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:32 2022] DELETE /identity/v3/projects/8360bdaafd854d6b8073e78af1328334 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:32.752070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627c0c7-4e66-47ea-bb05-9bb2689c4470 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:32.753073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627c0c7-4e66-47ea-bb05-9bb2689c4470 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:32.753536 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627c0c7-4e66-47ea-bb05-9bb2689c4470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:32.817070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3627c0c7-4e66-47ea-bb05-9bb2689c4470 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:32.820369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3627c0c7-4e66-47ea-bb05-9bb2689c4470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:32.995588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 907/1823] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:32.999259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3977acd-a940-41e3-92b4-bd08075e9fa7 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:33.000224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3977acd-a940-41e3-92b4-bd08075e9fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:33.000502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3977acd-a940-41e3-92b4-bd08075e9fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:33.039077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-d3977acd-a940-41e3-92b4-bd08075e9fa7 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:33.042954 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d3977acd-a940-41e3-92b4-bd08075e9fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:33.068780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 917/1824] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:33.071626 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627c0c7-4e66-47ea-bb05-9bb2689c4470 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:33.084460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627c0c7-4e66-47ea-bb05-9bb2689c4470 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:33.096361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3339bf6-ab7b-4c3b-a679-98c3632cdfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:33.096992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3339bf6-ab7b-4c3b-a679-98c3632cdfdb admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:33.097338 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3339bf6-ab7b-4c3b-a679-98c3632cdfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:33.097612 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3339bf6-ab7b-4c3b-a679-98c3632cdfdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:33.098263 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3339bf6-ab7b-4c3b-a679-98c3632cdfdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:33.098563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3339bf6-ab7b-4c3b-a679-98c3632cdfdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:33.100351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3339bf6-ab7b-4c3b-a679-98c3632cdfdb admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:33.131992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 908/1825] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:07:33.133124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3977acd-a940-41e3-92b4-bd08075e9fa7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:33.148644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3977acd-a940-41e3-92b4-bd08075e9fa7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:33.172434 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723e501-2887-43f9-8900-1e169138d24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:33.173375 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723e501-2887-43f9-8900-1e169138d24b admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:33.173873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723e501-2887-43f9-8900-1e169138d24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:33.174357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723e501-2887-43f9-8900-1e169138d24b admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:33.175245 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723e501-2887-43f9-8900-1e169138d24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:33.177390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723e501-2887-43f9-8900-1e169138d24b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:33.211739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:33.244170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 918/1826] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:33 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:33.248066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3339bf6-ab7b-4c3b-a679-98c3632cdfdb admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:33.266407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3339bf6-ab7b-4c3b-a679-98c3632cdfdb admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:33.356982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:33.450110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dfd06-bb1d-463d-a916-157aae46b717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:33.467311 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dfd06-bb1d-463d-a916-157aae46b717 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:33.468072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dfd06-bb1d-463d-a916-157aae46b717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:33.468701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dfd06-bb1d-463d-a916-157aae46b717 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:33.469868 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916dfd06-bb1d-463d-a916-157aae46b717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:33.472375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916dfd06-bb1d-463d-a916-157aae46b717 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:33.546848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 909/1827] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:33 2022] POST /identity/v3/users => generated 296 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:33.550707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723e501-2887-43f9-8900-1e169138d24b admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:33.580087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723e501-2887-43f9-8900-1e169138d24b admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:33.586426 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d0c6d-dcc4-495c-a9ed-c4774649fca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:33.587373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d0c6d-dcc4-495c-a9ed-c4774649fca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:33.587898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d0c6d-dcc4-495c-a9ed-c4774649fca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:33.590664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d0c6d-dcc4-495c-a9ed-c4774649fca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:33.591669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d0c6d-dcc4-495c-a9ed-c4774649fca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:33.593936 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d0c6d-dcc4-495c-a9ed-c4774649fca8 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:33.622878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dfd06-bb1d-463d-a916-157aae46b717 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:33.634650 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 919/1828] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:33.655472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dfd06-bb1d-463d-a916-157aae46b717 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:33.675885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae391d-3ccd-437a-9f80-125145172e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:33.680546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae391d-3ccd-437a-9f80-125145172e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:33.681611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae391d-3ccd-437a-9f80-125145172e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:33.683653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae391d-3ccd-437a-9f80-125145172e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:33.684777 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae391d-3ccd-437a-9f80-125145172e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:33.694812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae391d-3ccd-437a-9f80-125145172e41 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:33.756520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 910/1829] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:33.762634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d0c6d-dcc4-495c-a9ed-c4774649fca8 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:33.778439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d0c6d-dcc4-495c-a9ed-c4774649fca8 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:33.802263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7e178-e332-4aef-9b30-476e1cfdaa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:33.803262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7e178-e332-4aef-9b30-476e1cfdaa42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:33.803660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7e178-e332-4aef-9b30-476e1cfdaa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:33.804045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7e178-e332-4aef-9b30-476e1cfdaa42 admin admin] PATH_INFO: `/v3/projects/a2615186ec19469fa59cf7547a3e35c4/users/7ef4cc4ea6514adc8542d76b76c5f632/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:33.862686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7e178-e332-4aef-9b30-476e1cfdaa42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2615186ec19469fa59cf7547a3e35c4, user_id=7ef4cc4ea6514adc8542d76b76c5f632, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:33.869329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7e178-e332-4aef-9b30-476e1cfdaa42 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:33.900977 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 920/1830] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:33 2022] PUT /identity/v3/projects/a2615186ec19469fa59cf7547a3e35c4/users/7ef4cc4ea6514adc8542d76b76c5f632/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:33.911964 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbad336-e400-4f76-b94d-db773b087e5d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:33.912345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbad336-e400-4f76-b94d-db773b087e5d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:33.912685 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbad336-e400-4f76-b94d-db773b087e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:33.991453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3bbad336-e400-4f76-b94d-db773b087e5d None None] MFA Rules not processed for user `7ef4cc4ea6514adc8542d76b76c5f632`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:33.992924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3bbad336-e400-4f76-b94d-db773b087e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:34.109141 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 911/1831] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:34.120515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7e178-e332-4aef-9b30-476e1cfdaa42 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:34.124988 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-74c7e178-e332-4aef-9b30-476e1cfdaa42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:34.255662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7e178-e332-4aef-9b30-476e1cfdaa42 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:34.271136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad12f64-28d7-4abd-85a7-1c2e19d49e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:34.271748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad12f64-28d7-4abd-85a7-1c2e19d49e83 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:34.272102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad12f64-28d7-4abd-85a7-1c2e19d49e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:34.272394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad12f64-28d7-4abd-85a7-1c2e19d49e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:34.310600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad12f64-28d7-4abd-85a7-1c2e19d49e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:34.312263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad12f64-28d7-4abd-85a7-1c2e19d49e83 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:34.322725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 921/1832] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:34.601149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbad336-e400-4f76-b94d-db773b087e5d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:34.602635 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3bbad336-e400-4f76-b94d-db773b087e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:34.818470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbad336-e400-4f76-b94d-db773b087e5d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:34.832900 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:34.850144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:34.851906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:34.852371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] PATH_INFO: `/v3/users/2d91c33ff20f4b8ab4d3ac9db0ab815c` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:34.858199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d91c33ff20f4b8ab4d3ac9db0ab815c)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:34.861016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:35.007566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d91c33ff20f4b8ab4d3ac9db0ab815c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:35.008786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d91c33ff20f4b8ab4d3ac9db0ab815c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:35.015671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:35.041238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d91c33ff20f4b8ab4d3ac9db0ab815c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:35.075674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d91c33ff20f4b8ab4d3ac9db0ab815c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:35.104073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d91c33ff20f4b8ab4d3ac9db0ab815c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:35.117740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 912/1833] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:34 2022] DELETE /identity/v3/users/2d91c33ff20f4b8ab4d3ac9db0ab815c => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:35.122405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067a146-dc09-402d-a6fa-011ecaab7a37 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:35.122929 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067a146-dc09-402d-a6fa-011ecaab7a37 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:35.123045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067a146-dc09-402d-a6fa-011ecaab7a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:35.189157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6067a146-dc09-402d-a6fa-011ecaab7a37 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:35.190822 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6067a146-dc09-402d-a6fa-011ecaab7a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:35.402066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 922/1834] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:35.410892 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:35.412968 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:35.563273 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f9641-1c61-4a91-9b98-96f60974a19f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:35.574504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc04a3-1642-442e-ac84-4d6006cff68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:35.575693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc04a3-1642-442e-ac84-4d6006cff68d service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:35.577057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc04a3-1642-442e-ac84-4d6006cff68d service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:35.578478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc04a3-1642-442e-ac84-4d6006cff68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:35.593886 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc04a3-1642-442e-ac84-4d6006cff68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:35.595919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc04a3-1642-442e-ac84-4d6006cff68d service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:35.640584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 913/1835] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:36.291029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067a146-dc09-402d-a6fa-011ecaab7a37 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:36.295233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6067a146-dc09-402d-a6fa-011ecaab7a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:36.303714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc04a3-1642-442e-ac84-4d6006cff68d service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:36.305763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-79bc04a3-1642-442e-ac84-4d6006cff68d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:36.329874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067a146-dc09-402d-a6fa-011ecaab7a37 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:36.340002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:36.341117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:36.341552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:36.341967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] PATH_INFO: `/v3/users/095a332a446b4c6a97257bb7c76f1886` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:36.343378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=095a332a446b4c6a97257bb7c76f1886)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:36.345482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:36.361476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc04a3-1642-442e-ac84-4d6006cff68d service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:36.368646 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:36.369775 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:36.370156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:36.370687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] PATH_INFO: `/v3/projects/3cda39058bee49879e071275e52e4f31` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:36.372078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cda39058bee49879e071275e52e4f31)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:36.375208 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:36.451302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095a332a446b4c6a97257bb7c76f1886'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.452049 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095a332a446b4c6a97257bb7c76f1886'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.469464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:36.478517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cda39058bee49879e071275e52e4f31', 'request_id': 'req-19184338-ccd7-48be-a1b6-4c25ff8647cc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.479202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095a332a446b4c6a97257bb7c76f1886'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.498008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cda39058bee49879e071275e52e4f31', 'request_id': 'req-19184338-ccd7-48be-a1b6-4c25ff8647cc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.500556 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095a332a446b4c6a97257bb7c76f1886'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.501485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095a332a446b4c6a97257bb7c76f1886'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.516335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cda39058bee49879e071275e52e4f31', 'request_id': 'req-19184338-ccd7-48be-a1b6-4c25ff8647cc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.528824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 923/1836] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:36 2022] DELETE /identity/v3/users/095a332a446b4c6a97257bb7c76f1886 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:36.535875 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] The token cache is being invalidate because project 3cda39058bee49879e071275e52e4f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:36.536377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.537264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.541941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 914/1837] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:36 2022] DELETE /identity/v3/projects/3cda39058bee49879e071275e52e4f31 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:36.543308 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:36.550072 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:36.638630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f696c6-d179-4620-924d-3cb23e41ed58 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:36.643602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:36.644249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:36.644529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:36.644800 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] PATH_INFO: `/v3/users/edea4cdd71c840c08a6bb65d6cd837bf` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:36.646705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edea4cdd71c840c08a6bb65d6cd837bf)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:36.648864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:36.737868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edea4cdd71c840c08a6bb65d6cd837bf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.739136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edea4cdd71c840c08a6bb65d6cd837bf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.761825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edea4cdd71c840c08a6bb65d6cd837bf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.783534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edea4cdd71c840c08a6bb65d6cd837bf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.786093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edea4cdd71c840c08a6bb65d6cd837bf'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:36.810400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 924/1838] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:36 2022] DELETE /identity/v3/users/edea4cdd71c840c08a6bb65d6cd837bf => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:36.846786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:36.848828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:36.966064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:36.976732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19184338-ccd7-48be-a1b6-4c25ff8647cc admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:37.082996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:37.083758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:37.084090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:37.084369 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] PATH_INFO: `/v3/projects/67626d51a9484e9c98d0a4758707b896` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:37.086096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67626d51a9484e9c98d0a4758707b896)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:37.088448 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:37.216961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67626d51a9484e9c98d0a4758707b896', 'request_id': 'req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.239637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67626d51a9484e9c98d0a4758707b896', 'request_id': 'req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.254717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:37.257102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:37.301985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67626d51a9484e9c98d0a4758707b896', 'request_id': 'req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.321313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e046a19-e25f-4d66-a53b-ab1a2e5835f5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:37.324204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] The token cache is being invalidate because project 67626d51a9484e9c98d0a4758707b896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:37.324637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.325279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea1e4597-12d7-4e25-bbe4-42027e8b8bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.331310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 915/1839] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:36 2022] DELETE /identity/v3/projects/67626d51a9484e9c98d0a4758707b896 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:37.332848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e30a0-94bf-4865-ba8f-b20a1fc9c906 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:37.333264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e30a0-94bf-4865-ba8f-b20a1fc9c906 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:37.333522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e30a0-94bf-4865-ba8f-b20a1fc9c906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:37.391685 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-628e30a0-94bf-4865-ba8f-b20a1fc9c906 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:37.392929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-628e30a0-94bf-4865-ba8f-b20a1fc9c906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:37.460095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:37.474670 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:37.475928 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:37.476597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:37.477789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] PATH_INFO: `/v3/projects/cf601d6e73984fb3ae2d6cdd19ee8f5c` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:37.479764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf601d6e73984fb3ae2d6cdd19ee8f5c)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:37.482096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:37.540926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 916/1840] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:37.551314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a044ab4-e297-494d-b767-bc6f0faeb2c1 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:37.551832 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a044ab4-e297-494d-b767-bc6f0faeb2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:37.551946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a044ab4-e297-494d-b767-bc6f0faeb2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:37.623499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf601d6e73984fb3ae2d6cdd19ee8f5c', 'request_id': 'req-6feaaa26-7af2-439d-aefa-41c25e11eba3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.642501 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-7a044ab4-e297-494d-b767-bc6f0faeb2c1 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:37.643748 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7a044ab4-e297-494d-b767-bc6f0faeb2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:37.644034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf601d6e73984fb3ae2d6cdd19ee8f5c', 'request_id': 'req-6feaaa26-7af2-439d-aefa-41c25e11eba3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.656324 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf601d6e73984fb3ae2d6cdd19ee8f5c', 'request_id': 'req-6feaaa26-7af2-439d-aefa-41c25e11eba3'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.684460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] The token cache is being invalidate because project cf601d6e73984fb3ae2d6cdd19ee8f5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:37.684981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:37.685708 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.690689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.699021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 925/1841] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:37 2022] DELETE /identity/v3/projects/cf601d6e73984fb3ae2d6cdd19ee8f5c => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:37.701707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:37.703467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:37.758867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 917/1842] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:37.763014 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a044ab4-e297-494d-b767-bc6f0faeb2c1 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:37.790591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a044ab4-e297-494d-b767-bc6f0faeb2c1 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:37.799799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5ad87-55ba-43b2-8bbc-44bf99989d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:37.800719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5ad87-55ba-43b2-8bbc-44bf99989d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:37.801174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5ad87-55ba-43b2-8bbc-44bf99989d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:37.801687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5ad87-55ba-43b2-8bbc-44bf99989d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:37.802673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5ad87-55ba-43b2-8bbc-44bf99989d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:37.803695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5ad87-55ba-43b2-8bbc-44bf99989d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:37.806486 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaaa26-7af2-439d-aefa-41c25e11eba3 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:37.808321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5ad87-55ba-43b2-8bbc-44bf99989d63 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:37.811852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:37.812638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:37.812994 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:37.813338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] PATH_INFO: `/v3/users/52c06b565d194c8085242f064e65b99b` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:37.814731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c06b565d194c8085242f064e65b99b)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:37.816814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:37.830372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 918/1843] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:07:37.832399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5ad87-55ba-43b2-8bbc-44bf99989d63 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:37.851770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5ad87-55ba-43b2-8bbc-44bf99989d63 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:37.860554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45519de-8cb1-4948-a5e3-ba1c663914f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:37.862035 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45519de-8cb1-4948-a5e3-ba1c663914f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:37.862421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45519de-8cb1-4948-a5e3-ba1c663914f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:37.862756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45519de-8cb1-4948-a5e3-ba1c663914f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:37.866418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45519de-8cb1-4948-a5e3-ba1c663914f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:37.868498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45519de-8cb1-4948-a5e3-ba1c663914f6 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:37.899757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c06b565d194c8085242f064e65b99b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.901094 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c06b565d194c8085242f064e65b99b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.918283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 919/1844] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:37 2022] POST /identity/v3/projects => generated 365 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:37.922591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45519de-8cb1-4948-a5e3-ba1c663914f6 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:37.924373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e45519de-8cb1-4948-a5e3-ba1c663914f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:37.932505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c06b565d194c8085242f064e65b99b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.948912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c06b565d194c8085242f064e65b99b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.951106 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c06b565d194c8085242f064e65b99b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:37.978284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 926/1845] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:37 2022] DELETE /identity/v3/users/52c06b565d194c8085242f064e65b99b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:38.056452 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45519de-8cb1-4948-a5e3-ba1c663914f6 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.169769 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43aed5-73a0-49f6-b7a3-f3d27d9bf8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.171134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43aed5-73a0-49f6-b7a3-f3d27d9bf8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.171134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43aed5-73a0-49f6-b7a3-f3d27d9bf8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.171432 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43aed5-73a0-49f6-b7a3-f3d27d9bf8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.172175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43aed5-73a0-49f6-b7a3-f3d27d9bf8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.175012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43aed5-73a0-49f6-b7a3-f3d27d9bf8c3 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.220701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 920/1846] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:37 2022] POST /identity/v3/users => generated 283 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:38.221625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.228538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:38.237598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92247a-d866-4e32-8f21-71fd16497cbf admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.246480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115983-a4ca-4315-9b75-49ab2577d70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.247382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115983-a4ca-4315-9b75-49ab2577d70a admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.247810 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115983-a4ca-4315-9b75-49ab2577d70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.248298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115983-a4ca-4315-9b75-49ab2577d70a admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.249102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115983-a4ca-4315-9b75-49ab2577d70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.251110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115983-a4ca-4315-9b75-49ab2577d70a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.269725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43aed5-73a0-49f6-b7a3-f3d27d9bf8c3 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.271019 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 927/1847] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:38.281654 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43aed5-73a0-49f6-b7a3-f3d27d9bf8c3 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.286477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964c8ac-cff9-4ca7-be1f-f7d9e1dc2b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.289058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964c8ac-cff9-4ca7-be1f-f7d9e1dc2b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.289416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964c8ac-cff9-4ca7-be1f-f7d9e1dc2b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.289694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964c8ac-cff9-4ca7-be1f-f7d9e1dc2b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.290416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964c8ac-cff9-4ca7-be1f-f7d9e1dc2b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.292126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964c8ac-cff9-4ca7-be1f-f7d9e1dc2b68 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.309980 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115983-a4ca-4315-9b75-49ab2577d70a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.312255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 921/1848] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:38.326285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115983-a4ca-4315-9b75-49ab2577d70a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.336057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712a267-a1f0-486a-8ceb-7f77f9c3781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.336905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712a267-a1f0-486a-8ceb-7f77f9c3781c admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.337837 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712a267-a1f0-486a-8ceb-7f77f9c3781c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.338230 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712a267-a1f0-486a-8ceb-7f77f9c3781c admin admin] PATH_INFO: `/v3/projects/9ee3e792cb2043c29e9c1ffe06f7b13c/users/f8a6efc604b245f2b4ac190f109664e6/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.376262 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964c8ac-cff9-4ca7-be1f-f7d9e1dc2b68 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.378213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3964c8ac-cff9-4ca7-be1f-f7d9e1dc2b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:38.401334 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964c8ac-cff9-4ca7-be1f-f7d9e1dc2b68 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.417571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.418470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.418915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.419365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] PATH_INFO: `/v3/projects/df17840a56d342df8cc3703be5c6dbcc` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.420326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712a267-a1f0-486a-8ceb-7f77f9c3781c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee3e792cb2043c29e9c1ffe06f7b13c, user_id=f8a6efc604b245f2b4ac190f109664e6, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.421393 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df17840a56d342df8cc3703be5c6dbcc)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.423414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.430268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712a267-a1f0-486a-8ceb-7f77f9c3781c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.449052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 928/1849] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:38 2022] PUT /identity/v3/projects/9ee3e792cb2043c29e9c1ffe06f7b13c/users/f8a6efc604b245f2b4ac190f109664e6/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:38.460859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04b31f-05ce-4a94-abb8-ff9426490d13 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.461379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04b31f-05ce-4a94-abb8-ff9426490d13 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.467280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04b31f-05ce-4a94-abb8-ff9426490d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.474776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:38.514088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df17840a56d342df8cc3703be5c6dbcc', 'request_id': 'req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:38.524679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df17840a56d342df8cc3703be5c6dbcc', 'request_id': 'req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:38.527261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-cc04b31f-05ce-4a94-abb8-ff9426490d13 None None] MFA Rules not processed for user `f8a6efc604b245f2b4ac190f109664e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:38.528500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cc04b31f-05ce-4a94-abb8-ff9426490d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:38.536492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df17840a56d342df8cc3703be5c6dbcc', 'request_id': 'req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:38.549004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] The token cache is being invalidate because project df17840a56d342df8cc3703be5c6dbcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:38.549298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:38.549991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e820f4a7-80a4-46ed-bb0f-002a3c9047e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:38.551917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 922/1850] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:38 2022] DELETE /identity/v3/projects/df17840a56d342df8cc3703be5c6dbcc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:38.561271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c162-5865-4dda-ab08-6d11538e462e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.562219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c162-5865-4dda-ab08-6d11538e462e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.562652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c162-5865-4dda-ab08-6d11538e462e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.585748 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-0b73c162-5865-4dda-ab08-6d11538e462e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:38.587178 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0b73c162-5865-4dda-ab08-6d11538e462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:38.677592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 929/1851] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:38.681984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04b31f-05ce-4a94-abb8-ff9426490d13 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.685283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cc04b31f-05ce-4a94-abb8-ff9426490d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:38.695069 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 923/1852] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:38.700554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6ba10-00f3-48bf-be9a-19746b4c14d9 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.700694 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04b31f-05ce-4a94-abb8-ff9426490d13 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.700858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6ba10-00f3-48bf-be9a-19746b4c14d9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.701133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6ba10-00f3-48bf-be9a-19746b4c14d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.707847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e9f2-9de1-458e-b5db-e15bf1d2f28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.708799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e9f2-9de1-458e-b5db-e15bf1d2f28c admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.709199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e9f2-9de1-458e-b5db-e15bf1d2f28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.709533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e9f2-9de1-458e-b5db-e15bf1d2f28c admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.710319 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e9f2-9de1-458e-b5db-e15bf1d2f28c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.712569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e9f2-9de1-458e-b5db-e15bf1d2f28c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.727324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-62c6ba10-00f3-48bf-be9a-19746b4c14d9 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:38.731682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-62c6ba10-00f3-48bf-be9a-19746b4c14d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:38.747360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 924/1853] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:38.748616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6ba10-00f3-48bf-be9a-19746b4c14d9 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.763478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6ba10-00f3-48bf-be9a-19746b4c14d9 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.765350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 930/1854] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:38 2022] POST /identity/v3/projects => generated 365 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:38.769438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e9f2-9de1-458e-b5db-e15bf1d2f28c admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.872250 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e9f2-9de1-458e-b5db-e15bf1d2f28c admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.884884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd7791-aef8-4473-9f1e-0bcc3347b5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.885494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd7791-aef8-4473-9f1e-0bcc3347b5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.885775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd7791-aef8-4473-9f1e-0bcc3347b5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.886047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd7791-aef8-4473-9f1e-0bcc3347b5a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.886950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd7791-aef8-4473-9f1e-0bcc3347b5a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.890116 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd7791-aef8-4473-9f1e-0bcc3347b5a9 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.893651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d2913-2253-42c5-99c8-ca6cb1528b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.894373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d2913-2253-42c5-99c8-ca6cb1528b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.894656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d2913-2253-42c5-99c8-ca6cb1528b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.895007 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d2913-2253-42c5-99c8-ca6cb1528b53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.895674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d2913-2253-42c5-99c8-ca6cb1528b53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:38.895978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d2913-2253-42c5-99c8-ca6cb1528b53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.897596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d2913-2253-42c5-99c8-ca6cb1528b53 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.911238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 925/1855] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:07:38.915848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d2913-2253-42c5-99c8-ca6cb1528b53 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.921432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 931/1856] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:38 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:38.921432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd7791-aef8-4473-9f1e-0bcc3347b5a9 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.930169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d2913-2253-42c5-99c8-ca6cb1528b53 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.931103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd7791-aef8-4473-9f1e-0bcc3347b5a9 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.936163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f461adf-1063-4904-9033-380b0463e426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.937337 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f461adf-1063-4904-9033-380b0463e426 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.937616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab35f2f-5c39-409d-a3bf-91201f25a1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.937764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f461adf-1063-4904-9033-380b0463e426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.937882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f461adf-1063-4904-9033-380b0463e426 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.938491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab35f2f-5c39-409d-a3bf-91201f25a1de admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.938602 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f461adf-1063-4904-9033-380b0463e426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.938937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab35f2f-5c39-409d-a3bf-91201f25a1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.939330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab35f2f-5c39-409d-a3bf-91201f25a1de admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.940074 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab35f2f-5c39-409d-a3bf-91201f25a1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:38.940451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f461adf-1063-4904-9033-380b0463e426 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.941979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab35f2f-5c39-409d-a3bf-91201f25a1de admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:38.956516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 932/1857] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:38.957525 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab35f2f-5c39-409d-a3bf-91201f25a1de admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.969485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab35f2f-5c39-409d-a3bf-91201f25a1de admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:38.974521 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570ac89-85e1-48ba-90fa-d1260e94914d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:38.975441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570ac89-85e1-48ba-90fa-d1260e94914d admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:38.975731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570ac89-85e1-48ba-90fa-d1260e94914d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:38.975995 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570ac89-85e1-48ba-90fa-d1260e94914d admin admin] PATH_INFO: `/v3/projects/3831586216094c89b782a8664155e1d1/users/955c1b6634ea4a20867b434659aa24bc/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:38.979966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 926/1858] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:38 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:38.981317 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f461adf-1063-4904-9033-380b0463e426 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:38.992614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:39.001759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:39.003458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570ac89-85e1-48ba-90fa-d1260e94914d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3831586216094c89b782a8664155e1d1, user_id=955c1b6634ea4a20867b434659aa24bc, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.007566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570ac89-85e1-48ba-90fa-d1260e94914d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.039550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 933/1859] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:38 2022] PUT /identity/v3/projects/3831586216094c89b782a8664155e1d1/users/955c1b6634ea4a20867b434659aa24bc/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:39.041834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570ac89-85e1-48ba-90fa-d1260e94914d admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:39.085408 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f461adf-1063-4904-9033-380b0463e426 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:39.130315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570ac89-85e1-48ba-90fa-d1260e94914d admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:39.135246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34460b4e-d7cb-47b4-b7dc-f450ead6f1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:39.135986 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34460b4e-d7cb-47b4-b7dc-f450ead6f1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.136625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34460b4e-d7cb-47b4-b7dc-f450ead6f1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.137102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34460b4e-d7cb-47b4-b7dc-f450ead6f1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.137846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34460b4e-d7cb-47b4-b7dc-f450ead6f1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.139996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34460b4e-d7cb-47b4-b7dc-f450ead6f1e6 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.154245 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 934/1860] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:39.156532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d4c75-10c2-4f36-87f3-43ab28254ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:39.157389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d4c75-10c2-4f36-87f3-43ab28254ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.157747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d4c75-10c2-4f36-87f3-43ab28254ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.158101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d4c75-10c2-4f36-87f3-43ab28254ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.158400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34460b4e-d7cb-47b4-b7dc-f450ead6f1e6 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:39.159184 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d4c75-10c2-4f36-87f3-43ab28254ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.161341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d4c75-10c2-4f36-87f3-43ab28254ac5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.172058 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34460b4e-d7cb-47b4-b7dc-f450ead6f1e6 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:39.178108 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc0b3-bab6-4a2e-9471-0d96a02aed9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:39.179092 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc0b3-bab6-4a2e-9471-0d96a02aed9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.179432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc0b3-bab6-4a2e-9471-0d96a02aed9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.179848 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc0b3-bab6-4a2e-9471-0d96a02aed9c admin admin] PATH_INFO: `/v3/projects/3831586216094c89b782a8664155e1d1/users/955c1b6634ea4a20867b434659aa24bc/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.182149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fc0b3-bab6-4a2e-9471-0d96a02aed9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3831586216094c89b782a8664155e1d1, user_id=955c1b6634ea4a20867b434659aa24bc, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.188298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fc0b3-bab6-4a2e-9471-0d96a02aed9c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.193202 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 927/1861] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:38 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:39.196906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d4c75-10c2-4f36-87f3-43ab28254ac5 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:39.205338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 935/1862] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:39 2022] PUT /identity/v3/projects/3831586216094c89b782a8664155e1d1/users/955c1b6634ea4a20867b434659aa24bc/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:39.207975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d4c75-10c2-4f36-87f3-43ab28254ac5 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:39.217314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c524bb-72a9-46ca-9edb-7d0848bdc41b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.218156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c524bb-72a9-46ca-9edb-7d0848bdc41b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.218467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c524bb-72a9-46ca-9edb-7d0848bdc41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.259493 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-56c524bb-72a9-46ca-9edb-7d0848bdc41b None None] MFA Rules not processed for user `955c1b6634ea4a20867b434659aa24bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:39.262035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-56c524bb-72a9-46ca-9edb-7d0848bdc41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:39.295502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ed436-1601-40e0-88f8-9661c096a687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:39.296996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ed436-1601-40e0-88f8-9661c096a687 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.297430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ed436-1601-40e0-88f8-9661c096a687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.297709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ed436-1601-40e0-88f8-9661c096a687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.298537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ed436-1601-40e0-88f8-9661c096a687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.300340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ed436-1601-40e0-88f8-9661c096a687 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.311332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 928/1863] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:39.317033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ed436-1601-40e0-88f8-9661c096a687 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:39.327335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ed436-1601-40e0-88f8-9661c096a687 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:39.332591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f317bc-ab24-4381-81ca-0887d5fe4be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:39.333188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f317bc-ab24-4381-81ca-0887d5fe4be8 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.333494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f317bc-ab24-4381-81ca-0887d5fe4be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.333764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f317bc-ab24-4381-81ca-0887d5fe4be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.334391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f317bc-ab24-4381-81ca-0887d5fe4be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.336023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f317bc-ab24-4381-81ca-0887d5fe4be8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.348750 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 929/1864] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:39.352810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f317bc-ab24-4381-81ca-0887d5fe4be8 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:39.366173 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f317bc-ab24-4381-81ca-0887d5fe4be8 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:39.367615 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 936/1865] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:39.375077 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1dad8-48f4-43de-ad4e-dd84193370db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:39.376139 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1dad8-48f4-43de-ad4e-dd84193370db admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.376427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1dad8-48f4-43de-ad4e-dd84193370db admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.376944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1dad8-48f4-43de-ad4e-dd84193370db admin admin] PATH_INFO: `/v3/projects/14869eb90eaf4e09a5b7dc99684755d2/users/145f69970a7b48c8a65ea0dfd4f6d968/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.380539 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c524bb-72a9-46ca-9edb-7d0848bdc41b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:39.381948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-56c524bb-72a9-46ca-9edb-7d0848bdc41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:39.412982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1dad8-48f4-43de-ad4e-dd84193370db admin admin] RBAC: Authorizing `identity:create_grant(project_id=14869eb90eaf4e09a5b7dc99684755d2, user_id=145f69970a7b48c8a65ea0dfd4f6d968, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.416048 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1dad8-48f4-43de-ad4e-dd84193370db admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.443802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 930/1866] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:39 2022] PUT /identity/v3/projects/14869eb90eaf4e09a5b7dc99684755d2/users/145f69970a7b48c8a65ea0dfd4f6d968/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:39.446302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8fd94-2b34-4374-b576-cc80a7b28827 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.446652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8fd94-2b34-4374-b576-cc80a7b28827 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.447091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8fd94-2b34-4374-b576-cc80a7b28827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.473437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:39.485524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b5a8fd94-2b34-4374-b576-cc80a7b28827 None None] MFA Rules not processed for user `145f69970a7b48c8a65ea0dfd4f6d968`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:39.488106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b5a8fd94-2b34-4374-b576-cc80a7b28827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:39.496990 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c524bb-72a9-46ca-9edb-7d0848bdc41b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:39.508186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:39.557387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6827fc8-ace4-40e5-9d76-e66de3b9b993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:39.558091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6827fc8-ace4-40e5-9d76-e66de3b9b993 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.558421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6827fc8-ace4-40e5-9d76-e66de3b9b993 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.558625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6827fc8-ace4-40e5-9d76-e66de3b9b993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.591199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 931/1867] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:39.611419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8fd94-2b34-4374-b576-cc80a7b28827 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:39.617478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b5a8fd94-2b34-4374-b576-cc80a7b28827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:39.641982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8fd94-2b34-4374-b576-cc80a7b28827 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:39.648219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ec77f-ae45-40b9-bdd0-8e4fcc6562ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:39.649325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ec77f-ae45-40b9-bdd0-8e4fcc6562ca service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.649609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ec77f-ae45-40b9-bdd0-8e4fcc6562ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.649879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ec77f-ae45-40b9-bdd0-8e4fcc6562ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.660393 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ec77f-ae45-40b9-bdd0-8e4fcc6562ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.661853 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ec77f-ae45-40b9-bdd0-8e4fcc6562ca service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.668440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6827fc8-ace4-40e5-9d76-e66de3b9b993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.670395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6827fc8-ace4-40e5-9d76-e66de3b9b993 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.689412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 932/1868] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:39.704979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 937/1869] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:07:39.821053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ec77f-ae45-40b9-bdd0-8e4fcc6562ca service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:39.831220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ec77f-ae45-40b9-bdd0-8e4fcc6562ca service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:39.838241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28b730-d824-4ce5-abd5-7f27f93a6915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:39.838966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28b730-d824-4ce5-abd5-7f27f93a6915 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:39.839350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28b730-d824-4ce5-abd5-7f27f93a6915 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:39.839693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28b730-d824-4ce5-abd5-7f27f93a6915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:39.928991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28b730-d824-4ce5-abd5-7f27f93a6915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:39.934157 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28b730-d824-4ce5-abd5-7f27f93a6915 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:39.954534 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 933/1870] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:45.255772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6827fc8-ace4-40e5-9d76-e66de3b9b993 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:45.366610 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6827fc8-ace4-40e5-9d76-e66de3b9b993 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:45.373296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:45.374160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:45.374624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:45.375240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] PATH_INFO: `/v3/users/f8a6efc604b245f2b4ac190f109664e6` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:45.376653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a6efc604b245f2b4ac190f109664e6)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:45.379425 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:45.469276 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a6efc604b245f2b4ac190f109664e6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:45.470001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a6efc604b245f2b4ac190f109664e6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:45.488794 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a6efc604b245f2b4ac190f109664e6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:45.490118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:45.499994 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a6efc604b245f2b4ac190f109664e6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:45.500739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a6efc604b245f2b4ac190f109664e6'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:45.514889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 938/1871] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:45 2022] DELETE /identity/v3/users/f8a6efc604b245f2b4ac190f109664e6 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:45.519451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df61d60-0c82-4d39-9054-250d23cb236e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:45.520881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df61d60-0c82-4d39-9054-250d23cb236e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:45.520881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df61d60-0c82-4d39-9054-250d23cb236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:45.544142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-8df61d60-0c82-4d39-9054-250d23cb236e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:45.545403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8df61d60-0c82-4d39-9054-250d23cb236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:45.616984 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 934/1872] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:45.623873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:45.625411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:45.697706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f899311-22da-4515-ba89-970b1e2bb4f1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:45.702633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349ecee-d61d-4312-a210-ec239759c9f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:45.703420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349ecee-d61d-4312-a210-ec239759c9f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:45.703704 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349ecee-d61d-4312-a210-ec239759c9f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:45.703976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349ecee-d61d-4312-a210-ec239759c9f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:45.714953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349ecee-d61d-4312-a210-ec239759c9f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:45.718746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349ecee-d61d-4312-a210-ec239759c9f9 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:45.730630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 939/1873] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:45.852952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678133b8-b116-4f26-b0a4-163fde29d84a None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:45.853243 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678133b8-b116-4f26-b0a4-163fde29d84a None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:45.853646 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678133b8-b116-4f26-b0a4-163fde29d84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:45.874424 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-678133b8-b116-4f26-b0a4-163fde29d84a None None] MFA Rules not processed for user `fec54d8c1ea24cc8bbbeb6fef1fd2028`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:45.878828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-678133b8-b116-4f26-b0a4-163fde29d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:45.879763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349ecee-d61d-4312-a210-ec239759c9f9 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:45.881231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f349ecee-d61d-4312-a210-ec239759c9f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:45.901073 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349ecee-d61d-4312-a210-ec239759c9f9 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:45.905971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:45.906821 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:45.907127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:45.907400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] PATH_INFO: `/v3/users/145f69970a7b48c8a65ea0dfd4f6d968` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:45.908808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=145f69970a7b48c8a65ea0dfd4f6d968)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:45.910728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:45.925185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:45.968437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '145f69970a7b48c8a65ea0dfd4f6d968'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:45.968999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '145f69970a7b48c8a65ea0dfd4f6d968'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:45.986810 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '145f69970a7b48c8a65ea0dfd4f6d968'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:45.997035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '145f69970a7b48c8a65ea0dfd4f6d968'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:45.998000 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '145f69970a7b48c8a65ea0dfd4f6d968'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.010616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 935/1874] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:46.025717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 940/1875] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:45 2022] DELETE /identity/v3/users/145f69970a7b48c8a65ea0dfd4f6d968 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:46.028066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6d0c0-3ccd-4806-87b4-0ff71b9775d2 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:46.028066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6d0c0-3ccd-4806-87b4-0ff71b9775d2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:46.028066 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6d0c0-3ccd-4806-87b4-0ff71b9775d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:46.046882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:46.048485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:46.050934 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-20f6d0c0-3ccd-4806-87b4-0ff71b9775d2 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:46.051825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-20f6d0c0-3ccd-4806-87b4-0ff71b9775d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:46.149743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:46.159604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc5f84-6a10-43c3-ba8d-9e8a65293ef8 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:46.170285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:46.171006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:46.172979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:46.172979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] PATH_INFO: `/v3/projects/9ee3e792cb2043c29e9c1ffe06f7b13c` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:46.174455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee3e792cb2043c29e9c1ffe06f7b13c)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:46.176586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:46.183495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 936/1876] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:46.194860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6d0c0-3ccd-4806-87b4-0ff71b9775d2 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:46.196375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-20f6d0c0-3ccd-4806-87b4-0ff71b9775d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:46.258292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee3e792cb2043c29e9c1ffe06f7b13c', 'request_id': 'req-20273f8c-a888-40ad-9112-2b6080c79e22'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.270017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6d0c0-3ccd-4806-87b4-0ff71b9775d2 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:46.270912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee3e792cb2043c29e9c1ffe06f7b13c', 'request_id': 'req-20273f8c-a888-40ad-9112-2b6080c79e22'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.280832 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee3e792cb2043c29e9c1ffe06f7b13c', 'request_id': 'req-20273f8c-a888-40ad-9112-2b6080c79e22'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.310657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] The token cache is being invalidate because project 9ee3e792cb2043c29e9c1ffe06f7b13c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:46.310999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.312587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.314823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 941/1877] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:46 2022] DELETE /identity/v3/projects/9ee3e792cb2043c29e9c1ffe06f7b13c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:46.319201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:46.320631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:46.334180 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:46.334881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:46.335188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:46.335554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:46.337444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:46.411722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20273f8c-a888-40ad-9112-2b6080c79e22 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:46.427068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:46.437190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:46.437848 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:46.438124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:46.438400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] PATH_INFO: `/v3/users/955c1b6634ea4a20867b434659aa24bc` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:46.439721 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=955c1b6634ea4a20867b434659aa24bc)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:46.441858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:46.464647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:46.466804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:46.484353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 937/1878] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:46.513836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '955c1b6634ea4a20867b434659aa24bc'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.514560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '955c1b6634ea4a20867b434659aa24bc'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.531975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '955c1b6634ea4a20867b434659aa24bc'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.540767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '955c1b6634ea4a20867b434659aa24bc'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.541566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '955c1b6634ea4a20867b434659aa24bc'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:46.570345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 942/1879] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:46 2022] DELETE /identity/v3/users/955c1b6634ea4a20867b434659aa24bc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:46.904253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:46.908999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:46.909601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:46.911468 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:46.932418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc73cb2-20e5-4b7d-b55e-f8363fa5f82d service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:46.938814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:47.005997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201bd74-ed12-4012-b51f-65965ec46a0c admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.012430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.013523 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.013764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.014221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.014354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.014592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] PATH_INFO: `/v3/projects/3831586216094c89b782a8664155e1d1` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.014713 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.014872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] PATH_INFO: `/v3/projects/14869eb90eaf4e09a5b7dc99684755d2` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.016096 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14869eb90eaf4e09a5b7dc99684755d2)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.018050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.019782 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3831586216094c89b782a8664155e1d1)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.022518 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.080568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3831586216094c89b782a8664155e1d1', 'request_id': 'req-d410bec2-ce21-4d05-9a97-a83a8028bd4d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.095136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14869eb90eaf4e09a5b7dc99684755d2', 'request_id': 'req-6cbbeb43-f1a4-425f-a318-7e7468ed0252'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.099667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3831586216094c89b782a8664155e1d1', 'request_id': 'req-d410bec2-ce21-4d05-9a97-a83a8028bd4d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.103830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14869eb90eaf4e09a5b7dc99684755d2', 'request_id': 'req-6cbbeb43-f1a4-425f-a318-7e7468ed0252'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.110693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3831586216094c89b782a8664155e1d1', 'request_id': 'req-d410bec2-ce21-4d05-9a97-a83a8028bd4d'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.118116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14869eb90eaf4e09a5b7dc99684755d2', 'request_id': 'req-6cbbeb43-f1a4-425f-a318-7e7468ed0252'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.125403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] The token cache is being invalidate because project 3831586216094c89b782a8664155e1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:47.125767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.126537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d410bec2-ce21-4d05-9a97-a83a8028bd4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.129872 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 943/1880] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:46 2022] DELETE /identity/v3/projects/3831586216094c89b782a8664155e1d1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:47.130286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] The token cache is being invalidate because project 14869eb90eaf4e09a5b7dc99684755d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:47.130760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.131479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6cbbeb43-f1a4-425f-a318-7e7468ed0252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:47.135401 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da62a8b-e582-44e0-99a3-a7cf7df37fe4 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.135882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da62a8b-e582-44e0-99a3-a7cf7df37fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.136602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da62a8b-e582-44e0-99a3-a7cf7df37fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.137036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 938/1881] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:46 2022] DELETE /identity/v3/projects/14869eb90eaf4e09a5b7dc99684755d2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:47.143060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa14cb-0bf1-4cd7-814d-e7685c9a0923 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.144837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa14cb-0bf1-4cd7-814d-e7685c9a0923 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.144837 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa14cb-0bf1-4cd7-814d-e7685c9a0923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.163615 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-4da62a8b-e582-44e0-99a3-a7cf7df37fe4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:47.165100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4da62a8b-e582-44e0-99a3-a7cf7df37fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:47.168921 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-93fa14cb-0bf1-4cd7-814d-e7685c9a0923 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:47.169678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-93fa14cb-0bf1-4cd7-814d-e7685c9a0923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:47.239789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 939/1882] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:47.242717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 944/1883] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:47.251362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5f590-0bc2-4b61-9b67-90a4826dad75 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.251643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5f590-0bc2-4b61-9b67-90a4826dad75 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.251917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5f590-0bc2-4b61-9b67-90a4826dad75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.252217 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6ad17-1645-43ea-adce-aac19d21f478 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.253086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6ad17-1645-43ea-adce-aac19d21f478 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.253218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6ad17-1645-43ea-adce-aac19d21f478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.273623 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-fca5f590-0bc2-4b61-9b67-90a4826dad75 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:47.274834 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-fca5f590-0bc2-4b61-9b67-90a4826dad75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:47.279302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e1f6ad17-1645-43ea-adce-aac19d21f478 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:47.281101 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e1f6ad17-1645-43ea-adce-aac19d21f478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:47.287456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 940/1884] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:47.291113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5f590-0bc2-4b61-9b67-90a4826dad75 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.292993 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 945/1885] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:47.296644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:47.297202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6ad17-1645-43ea-adce-aac19d21f478 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.304360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5f590-0bc2-4b61-9b67-90a4826dad75 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.306213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6ad17-1645-43ea-adce-aac19d21f478 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.310921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6226d0e-56bf-4bf8-82de-e05e34e14736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.311594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6226d0e-56bf-4bf8-82de-e05e34e14736 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.311758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2a126-394a-4225-9bd7-fef2e18294e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.311891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6226d0e-56bf-4bf8-82de-e05e34e14736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.312125 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6226d0e-56bf-4bf8-82de-e05e34e14736 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.312368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2a126-394a-4225-9bd7-fef2e18294e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.312650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2a126-394a-4225-9bd7-fef2e18294e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.312761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6226d0e-56bf-4bf8-82de-e05e34e14736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:47.312949 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2a126-394a-4225-9bd7-fef2e18294e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.313066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6226d0e-56bf-4bf8-82de-e05e34e14736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.313597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2a126-394a-4225-9bd7-fef2e18294e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:47.313903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2a126-394a-4225-9bd7-fef2e18294e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.314627 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6226d0e-56bf-4bf8-82de-e05e34e14736 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.315567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2a126-394a-4225-9bd7-fef2e18294e2 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.328809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 946/1886] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:07:47.329807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 941/1887] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:07:47.332500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6226d0e-56bf-4bf8-82de-e05e34e14736 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.333828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2a126-394a-4225-9bd7-fef2e18294e2 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.345285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2a126-394a-4225-9bd7-fef2e18294e2 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.345457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6226d0e-56bf-4bf8-82de-e05e34e14736 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.350591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf46a2-bc3d-4b1f-9ce8-b55922f3ef3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.351266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf46a2-bc3d-4b1f-9ce8-b55922f3ef3b admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.351679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf46a2-bc3d-4b1f-9ce8-b55922f3ef3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.351958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf46a2-bc3d-4b1f-9ce8-b55922f3ef3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.352590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf46a2-bc3d-4b1f-9ce8-b55922f3ef3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.354521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf46a2-bc3d-4b1f-9ce8-b55922f3ef3b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.355682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88445994-c831-4a75-818b-05409cf85d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.356723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88445994-c831-4a75-818b-05409cf85d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.356723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88445994-c831-4a75-818b-05409cf85d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.356917 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88445994-c831-4a75-818b-05409cf85d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.357534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88445994-c831-4a75-818b-05409cf85d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.359687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88445994-c831-4a75-818b-05409cf85d10 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.378420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:47.392463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 942/1888] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:47.397451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 947/1889] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:47.397583 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf46a2-bc3d-4b1f-9ce8-b55922f3ef3b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.398440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88445994-c831-4a75-818b-05409cf85d10 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.491872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf46a2-bc3d-4b1f-9ce8-b55922f3ef3b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.526626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88445994-c831-4a75-818b-05409cf85d10 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.540496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fb72d-07d9-4cbc-a34d-3d92102e16ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.541035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fb72d-07d9-4cbc-a34d-3d92102e16ee admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.541338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fb72d-07d9-4cbc-a34d-3d92102e16ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.541578 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fb72d-07d9-4cbc-a34d-3d92102e16ee admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.542208 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452fb72d-07d9-4cbc-a34d-3d92102e16ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.544219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452fb72d-07d9-4cbc-a34d-3d92102e16ee admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.571034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 948/1890] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:47.573819 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fb72d-07d9-4cbc-a34d-3d92102e16ee admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.587194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fb72d-07d9-4cbc-a34d-3d92102e16ee admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.591903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f74f0c-6fc9-4e84-af57-8251914f0dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.592070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac8567-e6d0-4b0e-ae74-06dd7cd412b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.592678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f74f0c-6fc9-4e84-af57-8251914f0dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.592790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac8567-e6d0-4b0e-ae74-06dd7cd412b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.592975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f74f0c-6fc9-4e84-af57-8251914f0dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.593090 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac8567-e6d0-4b0e-ae74-06dd7cd412b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.593283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f74f0c-6fc9-4e84-af57-8251914f0dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.593387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac8567-e6d0-4b0e-ae74-06dd7cd412b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.593843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f74f0c-6fc9-4e84-af57-8251914f0dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.594040 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac8567-e6d0-4b0e-ae74-06dd7cd412b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.595663 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac8567-e6d0-4b0e-ae74-06dd7cd412b9 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.595857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f74f0c-6fc9-4e84-af57-8251914f0dd3 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.606689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 949/1891] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:47.611213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac8567-e6d0-4b0e-ae74-06dd7cd412b9 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.617800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 943/1892] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:47.621721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f74f0c-6fc9-4e84-af57-8251914f0dd3 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.621972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac8567-e6d0-4b0e-ae74-06dd7cd412b9 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.627462 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eb301-3120-407f-abf6-70968089c208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.628254 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eb301-3120-407f-abf6-70968089c208 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.628533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eb301-3120-407f-abf6-70968089c208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.628803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eb301-3120-407f-abf6-70968089c208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.630008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eb301-3120-407f-abf6-70968089c208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.632080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eb301-3120-407f-abf6-70968089c208 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.632857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f74f0c-6fc9-4e84-af57-8251914f0dd3 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.638944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6d52-f5ba-4b0e-818a-15a3299b13be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.640129 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6d52-f5ba-4b0e-818a-15a3299b13be admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.640414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6d52-f5ba-4b0e-818a-15a3299b13be admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.640687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6d52-f5ba-4b0e-818a-15a3299b13be admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.641471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6d52-f5ba-4b0e-818a-15a3299b13be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.643168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6d52-f5ba-4b0e-818a-15a3299b13be admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.648552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 950/1893] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:47.649661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eb301-3120-407f-abf6-70968089c208 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.656847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 944/1894] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:47.657624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6d52-f5ba-4b0e-818a-15a3299b13be admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.660587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eb301-3120-407f-abf6-70968089c208 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.666293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e9859-6883-41f2-82c9-65cf317abc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.666946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e9859-6883-41f2-82c9-65cf317abc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.667259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e9859-6883-41f2-82c9-65cf317abc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.667422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6d52-f5ba-4b0e-818a-15a3299b13be admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.667517 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e9859-6883-41f2-82c9-65cf317abc69 admin admin] PATH_INFO: `/v3/projects/d802b8ce56e046d6af6058d14b5e1ac1/users/fa16c5a0cd154c45938aff7a0ce987f1/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.672090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811262b9-14b3-414d-9939-fe6f752b6c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.672667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811262b9-14b3-414d-9939-fe6f752b6c91 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.672947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811262b9-14b3-414d-9939-fe6f752b6c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.673220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811262b9-14b3-414d-9939-fe6f752b6c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.673855 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811262b9-14b3-414d-9939-fe6f752b6c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.675485 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811262b9-14b3-414d-9939-fe6f752b6c91 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.681933 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e9859-6883-41f2-82c9-65cf317abc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d802b8ce56e046d6af6058d14b5e1ac1, user_id=fa16c5a0cd154c45938aff7a0ce987f1, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.684641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e9859-6883-41f2-82c9-65cf317abc69 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.686569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 945/1895] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:47.691861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811262b9-14b3-414d-9939-fe6f752b6c91 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.699183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 951/1896] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:47 2022] PUT /identity/v3/projects/d802b8ce56e046d6af6058d14b5e1ac1/users/fa16c5a0cd154c45938aff7a0ce987f1/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:47.701688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811262b9-14b3-414d-9939-fe6f752b6c91 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.710905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83a03c-0c49-4b14-b577-b55eac6f675f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.711220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83a03c-0c49-4b14-b577-b55eac6f675f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.711494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83a03c-0c49-4b14-b577-b55eac6f675f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.740692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-0b83a03c-0c49-4b14-b577-b55eac6f675f None None] MFA Rules not processed for user `fa16c5a0cd154c45938aff7a0ce987f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:47.741909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0b83a03c-0c49-4b14-b577-b55eac6f675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:47.745173 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:47.773259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10d4b2-7dd5-4a88-b374-48828be3c6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.775184 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10d4b2-7dd5-4a88-b374-48828be3c6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.775657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10d4b2-7dd5-4a88-b374-48828be3c6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.776115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10d4b2-7dd5-4a88-b374-48828be3c6d5 admin admin] PATH_INFO: `/v3/projects/79d35bd758904b888f67847522560758/users/3388875b82594451988646687c201091/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.793419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10d4b2-7dd5-4a88-b374-48828be3c6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d35bd758904b888f67847522560758, user_id=3388875b82594451988646687c201091, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:47.796196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10d4b2-7dd5-4a88-b374-48828be3c6d5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:47.811490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 946/1897] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:47 2022] PUT /identity/v3/projects/79d35bd758904b888f67847522560758/users/3388875b82594451988646687c201091/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:47.823166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 952/1898] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:47.823401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e034c1a-d33e-4f8c-b131-bcedc2b1ceaf None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.823590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e034c1a-d33e-4f8c-b131-bcedc2b1ceaf None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.823748 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e034c1a-d33e-4f8c-b131-bcedc2b1ceaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:47.837249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83a03c-0c49-4b14-b577-b55eac6f675f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.839849 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0b83a03c-0c49-4b14-b577-b55eac6f675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:47.854452 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-9e034c1a-d33e-4f8c-b131-bcedc2b1ceaf None None] MFA Rules not processed for user `3388875b82594451988646687c201091`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:47.855713 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9e034c1a-d33e-4f8c-b131-bcedc2b1ceaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:47.877735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:47.910075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83a03c-0c49-4b14-b577-b55eac6f675f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:47.933279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 947/1899] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:47.938256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e034c1a-d33e-4f8c-b131-bcedc2b1ceaf None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:47.968146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc518ff-e025-45af-9536-5f0ad4f8308b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:47.969211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc518ff-e025-45af-9536-5f0ad4f8308b service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:47.969667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc518ff-e025-45af-9536-5f0ad4f8308b service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:47.970252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc518ff-e025-45af-9536-5f0ad4f8308b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.005796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e034c1a-d33e-4f8c-b131-bcedc2b1ceaf None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.012833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbf09a-cb60-495b-92c1-1bd0d69140b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.013594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbf09a-cb60-495b-92c1-1bd0d69140b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.013947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbf09a-cb60-495b-92c1-1bd0d69140b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.014348 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbf09a-cb60-495b-92c1-1bd0d69140b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.015789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbf09a-cb60-495b-92c1-1bd0d69140b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.018308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbf09a-cb60-495b-92c1-1bd0d69140b8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.047459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc518ff-e025-45af-9536-5f0ad4f8308b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.048975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc518ff-e025-45af-9536-5f0ad4f8308b service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.054510 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 948/1900] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:47 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:07:48.055606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbf09a-cb60-495b-92c1-1bd0d69140b8 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.058638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 953/1901] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:48.105532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:48.137492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbf09a-cb60-495b-92c1-1bd0d69140b8 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.144241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb00a8-ed51-4c74-8f6b-3c004cf9e59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.144952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb00a8-ed51-4c74-8f6b-3c004cf9e59c admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.145118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb00a8-ed51-4c74-8f6b-3c004cf9e59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.145388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb00a8-ed51-4c74-8f6b-3c004cf9e59c admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.146012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb00a8-ed51-4c74-8f6b-3c004cf9e59c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.148072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb00a8-ed51-4c74-8f6b-3c004cf9e59c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.176083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 949/1902] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:48 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:48.178063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc518ff-e025-45af-9536-5f0ad4f8308b service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.189586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc518ff-e025-45af-9536-5f0ad4f8308b service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.197951 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5033afb-fbd8-40aa-8446-0719f85c6b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.198593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5033afb-fbd8-40aa-8446-0719f85c6b83 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.198897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5033afb-fbd8-40aa-8446-0719f85c6b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.199441 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5033afb-fbd8-40aa-8446-0719f85c6b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.199979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5033afb-fbd8-40aa-8446-0719f85c6b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.201699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5033afb-fbd8-40aa-8446-0719f85c6b83 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.227068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 954/1903] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:48.231915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb00a8-ed51-4c74-8f6b-3c004cf9e59c admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.244235 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb00a8-ed51-4c74-8f6b-3c004cf9e59c admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.248920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa24a9-188f-409e-8317-e76af02ba63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.249518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa24a9-188f-409e-8317-e76af02ba63e admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.249799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa24a9-188f-409e-8317-e76af02ba63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.250073 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa24a9-188f-409e-8317-e76af02ba63e admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.250764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa24a9-188f-409e-8317-e76af02ba63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.252612 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa24a9-188f-409e-8317-e76af02ba63e admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.264003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 950/1904] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:48.268287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5033afb-fbd8-40aa-8446-0719f85c6b83 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.279333 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5033afb-fbd8-40aa-8446-0719f85c6b83 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.284552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e65e7-3fb1-48a8-978b-1c69b1a03667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.285252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e65e7-3fb1-48a8-978b-1c69b1a03667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.285609 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e65e7-3fb1-48a8-978b-1c69b1a03667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.285920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e65e7-3fb1-48a8-978b-1c69b1a03667 admin admin] PATH_INFO: `/v3/projects/82aef298ec094370b2049c04dc9fbc53/users/c36df8312ffd401e97a11780743a3dab/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.302661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e65e7-3fb1-48a8-978b-1c69b1a03667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82aef298ec094370b2049c04dc9fbc53, user_id=c36df8312ffd401e97a11780743a3dab, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.305749 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e65e7-3fb1-48a8-978b-1c69b1a03667 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.319783 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 955/1905] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:48 2022] PUT /identity/v3/projects/82aef298ec094370b2049c04dc9fbc53/users/c36df8312ffd401e97a11780743a3dab/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:48.329936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5680785-4751-4337-a0e3-573f9736ea52 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.330222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5680785-4751-4337-a0e3-573f9736ea52 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.330522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5680785-4751-4337-a0e3-573f9736ea52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.377287 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b5680785-4751-4337-a0e3-573f9736ea52 None None] MFA Rules not processed for user `c36df8312ffd401e97a11780743a3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:48.378578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b5680785-4751-4337-a0e3-573f9736ea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:48.449418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 951/1906] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:48.454089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e65e7-3fb1-48a8-978b-1c69b1a03667 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.542688 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e65e7-3fb1-48a8-978b-1c69b1a03667 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.549396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2880f-f7f2-40ed-a940-f363e5386371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.550022 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2880f-f7f2-40ed-a940-f363e5386371 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.550282 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2880f-f7f2-40ed-a940-f363e5386371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.550579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2880f-f7f2-40ed-a940-f363e5386371 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.551481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2880f-f7f2-40ed-a940-f363e5386371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.553247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2880f-f7f2-40ed-a940-f363e5386371 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.563965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:48.585005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 956/1907] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:48 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:48.589037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5680785-4751-4337-a0e3-573f9736ea52 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.600182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5680785-4751-4337-a0e3-573f9736ea52 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.666545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb5d0b-1f5e-4fcf-9a19-63aa496ab5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.667312 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb5d0b-1f5e-4fcf-9a19-63aa496ab5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.667595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb5d0b-1f5e-4fcf-9a19-63aa496ab5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.667868 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb5d0b-1f5e-4fcf-9a19-63aa496ab5b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.668517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb5d0b-1f5e-4fcf-9a19-63aa496ab5b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.670499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb5d0b-1f5e-4fcf-9a19-63aa496ab5b0 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.693165 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 952/1908] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:48 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:48.697178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2880f-f7f2-40ed-a940-f363e5386371 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.707958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2880f-f7f2-40ed-a940-f363e5386371 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.712766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1270-3022-499d-b06e-1e40b4d1c7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.713444 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1270-3022-499d-b06e-1e40b4d1c7bb admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.713794 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1270-3022-499d-b06e-1e40b4d1c7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.714140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1270-3022-499d-b06e-1e40b4d1c7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.714885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d1270-3022-499d-b06e-1e40b4d1c7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.716566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d1270-3022-499d-b06e-1e40b4d1c7bb admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.727741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 957/1909] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:07:48.731902 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb5d0b-1f5e-4fcf-9a19-63aa496ab5b0 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.742474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb5d0b-1f5e-4fcf-9a19-63aa496ab5b0 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.747209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04fd28-fcdb-405b-84b1-f0b472938dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.747872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04fd28-fcdb-405b-84b1-f0b472938dec admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.748152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04fd28-fcdb-405b-84b1-f0b472938dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.748499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04fd28-fcdb-405b-84b1-f0b472938dec admin admin] PATH_INFO: `/v3/projects/9d2e838b0bdc4498b88941743ac20569/users/1a1896f1ec6c44bfbe0754f003ad5cd9/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.767616 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04fd28-fcdb-405b-84b1-f0b472938dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2e838b0bdc4498b88941743ac20569, user_id=1a1896f1ec6c44bfbe0754f003ad5cd9, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.770293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04fd28-fcdb-405b-84b1-f0b472938dec admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.783520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 953/1910] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:48 2022] PUT /identity/v3/projects/9d2e838b0bdc4498b88941743ac20569/users/1a1896f1ec6c44bfbe0754f003ad5cd9/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:48.787502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1270-3022-499d-b06e-1e40b4d1c7bb admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.798870 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1270-3022-499d-b06e-1e40b4d1c7bb admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.858396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639a88c-4935-41b5-b41d-0d52e8468499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.859026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639a88c-4935-41b5-b41d-0d52e8468499 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.859312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639a88c-4935-41b5-b41d-0d52e8468499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.859583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639a88c-4935-41b5-b41d-0d52e8468499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.860222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639a88c-4935-41b5-b41d-0d52e8468499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.861820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639a88c-4935-41b5-b41d-0d52e8468499 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.871399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 958/1911] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:48.875621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04fd28-fcdb-405b-84b1-f0b472938dec admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:48.886002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04fd28-fcdb-405b-84b1-f0b472938dec admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:48.891776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f749d4-0201-4c4e-98c9-bdfa52a76c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:48.892448 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f749d4-0201-4c4e-98c9-bdfa52a76c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.892725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f749d4-0201-4c4e-98c9-bdfa52a76c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.892995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f749d4-0201-4c4e-98c9-bdfa52a76c1d admin admin] PATH_INFO: `/v3/projects/9d2e838b0bdc4498b88941743ac20569/users/1a1896f1ec6c44bfbe0754f003ad5cd9/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.895092 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f749d4-0201-4c4e-98c9-bdfa52a76c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2e838b0bdc4498b88941743ac20569, user_id=1a1896f1ec6c44bfbe0754f003ad5cd9, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:48.897653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f749d4-0201-4c4e-98c9-bdfa52a76c1d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:48.903963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:48.910171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 954/1912] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:48 2022] PUT /identity/v3/projects/9d2e838b0bdc4498b88941743ac20569/users/1a1896f1ec6c44bfbe0754f003ad5cd9/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:07:48.920951 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9fa0-2e1f-4d0b-98f4-5fc61f8f5d34 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:48.921279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9fa0-2e1f-4d0b-98f4-5fc61f8f5d34 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:48.921508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9fa0-2e1f-4d0b-98f4-5fc61f8f5d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:48.950145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-779f9fa0-2e1f-4d0b-98f4-5fc61f8f5d34 None None] MFA Rules not processed for user `1a1896f1ec6c44bfbe0754f003ad5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:48.951402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-779f9fa0-2e1f-4d0b-98f4-5fc61f8f5d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:49.019644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 959/1913] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:49.031390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f749d4-0201-4c4e-98c9-bdfa52a76c1d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:49.087207 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f749d4-0201-4c4e-98c9-bdfa52a76c1d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:49.090737 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b80381-e004-443f-a6dc-80e2d064e236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:49.091367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b80381-e004-443f-a6dc-80e2d064e236 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:49.091644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b80381-e004-443f-a6dc-80e2d064e236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:49.091912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b80381-e004-443f-a6dc-80e2d064e236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:49.147656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b80381-e004-443f-a6dc-80e2d064e236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:49.149079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b80381-e004-443f-a6dc-80e2d064e236 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:49.158761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 955/1914] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:07:53.743038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd053a99-8742-4dca-a185-74fc9f57bff0 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:53.743499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd053a99-8742-4dca-a185-74fc9f57bff0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:53.743681 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd053a99-8742-4dca-a185-74fc9f57bff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:53.775198 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-bd053a99-8742-4dca-a185-74fc9f57bff0 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:53.776480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-bd053a99-8742-4dca-a185-74fc9f57bff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:53.895281 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 960/1915] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:07:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:53.907068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b80381-e004-443f-a6dc-80e2d064e236 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:53.908862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c8b80381-e004-443f-a6dc-80e2d064e236 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:54.053320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b80381-e004-443f-a6dc-80e2d064e236 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:54.073851 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12650f-9bf9-4527-bbdc-7a33f6709f51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:54.074901 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12650f-9bf9-4527-bbdc-7a33f6709f51 service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:54.075304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12650f-9bf9-4527-bbdc-7a33f6709f51 service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:54.075747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12650f-9bf9-4527-bbdc-7a33f6709f51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:54.093958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12650f-9bf9-4527-bbdc-7a33f6709f51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:54.096040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12650f-9bf9-4527-bbdc-7a33f6709f51 service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:54.108549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 956/1916] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:07:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:07:54.148624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd053a99-8742-4dca-a185-74fc9f57bff0 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:54.155647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:54.162316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd053a99-8742-4dca-a185-74fc9f57bff0 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:54.167095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaada6-d5b6-4bd1-a537-1cb65a4de856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:54.167681 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaada6-d5b6-4bd1-a537-1cb65a4de856 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:54.167999 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaada6-d5b6-4bd1-a537-1cb65a4de856 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:54.168473 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaada6-d5b6-4bd1-a537-1cb65a4de856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:54.300275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaada6-d5b6-4bd1-a537-1cb65a4de856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:54.304897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaada6-d5b6-4bd1-a537-1cb65a4de856 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:54.330295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 961/1917] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:55.646364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12650f-9bf9-4527-bbdc-7a33f6709f51 service swift] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:55.648249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-fe12650f-9bf9-4527-bbdc-7a33f6709f51 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:55.669786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12650f-9bf9-4527-bbdc-7a33f6709f51 service swift] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:55.682539 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a260e1-b535-4d8d-b780-eb40ee59ec25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:55.683188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a260e1-b535-4d8d-b780-eb40ee59ec25 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:55.683462 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a260e1-b535-4d8d-b780-eb40ee59ec25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:55.683733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a260e1-b535-4d8d-b780-eb40ee59ec25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:55.706368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a260e1-b535-4d8d-b780-eb40ee59ec25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:55.709072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a260e1-b535-4d8d-b780-eb40ee59ec25 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:55.724713 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 957/1918] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:56.501295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaada6-d5b6-4bd1-a537-1cb65a4de856 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:56.503405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f3eaada6-d5b6-4bd1-a537-1cb65a4de856 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:56.655349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaada6-d5b6-4bd1-a537-1cb65a4de856 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:56.662016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:56.663051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:56.663532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:56.663964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] PATH_INFO: `/v3/users/7ef4cc4ea6514adc8542d76b76c5f632` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:56.665285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ef4cc4ea6514adc8542d76b76c5f632)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:56.667629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:56.785167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ef4cc4ea6514adc8542d76b76c5f632'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:56.785914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ef4cc4ea6514adc8542d76b76c5f632'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:56.816146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ef4cc4ea6514adc8542d76b76c5f632'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:56.828175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ef4cc4ea6514adc8542d76b76c5f632'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:56.828709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ef4cc4ea6514adc8542d76b76c5f632'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:56.873537 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 962/1919] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:07:56 2022] DELETE /identity/v3/users/7ef4cc4ea6514adc8542d76b76c5f632 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:56.882040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d483c37-228f-45ff-82b0-0563b2f194e9 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:56.882487 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d483c37-228f-45ff-82b0-0563b2f194e9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:56.882652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d483c37-228f-45ff-82b0-0563b2f194e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:56.950218 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-5d483c37-228f-45ff-82b0-0563b2f194e9 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:56.955331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5d483c37-228f-45ff-82b0-0563b2f194e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:57.007846 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:57.123995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 958/1920] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:57.133021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:57.135314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:57.178538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:57.280960 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d813d07-ff22-4186-9c5e-67c71be17037 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:57.295056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad65462-085b-4d02-922f-26ac0eb322c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:57.296787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad65462-085b-4d02-922f-26ac0eb322c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:57.298307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad65462-085b-4d02-922f-26ac0eb322c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:57.299249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad65462-085b-4d02-922f-26ac0eb322c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:57.338435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad65462-085b-4d02-922f-26ac0eb322c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:57.344572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad65462-085b-4d02-922f-26ac0eb322c1 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:57.364936 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 963/1921] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:07:58.117759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d483c37-228f-45ff-82b0-0563b2f194e9 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:58.125390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5d483c37-228f-45ff-82b0-0563b2f194e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:58.160525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d483c37-228f-45ff-82b0-0563b2f194e9 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:58.168013 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:58.169284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:58.180298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:58.180814 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] PATH_INFO: `/v3/projects/a2615186ec19469fa59cf7547a3e35c4` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:58.182761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2615186ec19469fa59cf7547a3e35c4)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:58.184938 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:58.272640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2615186ec19469fa59cf7547a3e35c4', 'request_id': 'req-ef6a98af-971b-464f-a817-b6704529b3f8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:58.282535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2615186ec19469fa59cf7547a3e35c4', 'request_id': 'req-ef6a98af-971b-464f-a817-b6704529b3f8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:58.292512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2615186ec19469fa59cf7547a3e35c4', 'request_id': 'req-ef6a98af-971b-464f-a817-b6704529b3f8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:58.314131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] The token cache is being invalidate because project a2615186ec19469fa59cf7547a3e35c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:07:58.314517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:58.315665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ef6a98af-971b-464f-a817-b6704529b3f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:07:58.317718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 959/1922] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:07:58 2022] DELETE /identity/v3/projects/a2615186ec19469fa59cf7547a3e35c4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:58.324021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e144c28-986c-4208-855a-83a4529d4400 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:58.324900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e144c28-986c-4208-855a-83a4529d4400 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:58.325492 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e144c28-986c-4208-855a-83a4529d4400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:58.349311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-5e144c28-986c-4208-855a-83a4529d4400 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:58.350642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5e144c28-986c-4208-855a-83a4529d4400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:58.434603 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 964/1923] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:58.442697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca25846-d8d6-4d9b-8194-1d853dc7671e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:58.443039 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca25846-d8d6-4d9b-8194-1d853dc7671e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:58.443449 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca25846-d8d6-4d9b-8194-1d853dc7671e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:58.475695 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3ca25846-d8d6-4d9b-8194-1d853dc7671e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:58.478797 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3ca25846-d8d6-4d9b-8194-1d853dc7671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:58.495842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 960/1924] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:07:58.500596 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e144c28-986c-4208-855a-83a4529d4400 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:58.521882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e144c28-986c-4208-855a-83a4529d4400 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:58.527097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22a391-0c4c-4e8a-9e78-bffcd1e692c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:58.527726 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22a391-0c4c-4e8a-9e78-bffcd1e692c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:58.528008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22a391-0c4c-4e8a-9e78-bffcd1e692c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:58.528430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22a391-0c4c-4e8a-9e78-bffcd1e692c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:58.529161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22a391-0c4c-4e8a-9e78-bffcd1e692c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:07:58.530456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22a391-0c4c-4e8a-9e78-bffcd1e692c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:58.532608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22a391-0c4c-4e8a-9e78-bffcd1e692c4 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:58.567003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 965/1925] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:07:58.567778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca25846-d8d6-4d9b-8194-1d853dc7671e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:58.577961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca25846-d8d6-4d9b-8194-1d853dc7671e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:58.587173 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fceda-04c2-499d-9c53-0641146a91ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:58.588006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fceda-04c2-499d-9c53-0641146a91ae admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:58.588358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fceda-04c2-499d-9c53-0641146a91ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:58.588700 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fceda-04c2-499d-9c53-0641146a91ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:58.589468 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fceda-04c2-499d-9c53-0641146a91ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:58.597238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fceda-04c2-499d-9c53-0641146a91ae admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:58.647401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 961/1926] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:07:58 2022] POST /identity/v3/projects => generated 399 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:58.652086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22a391-0c4c-4e8a-9e78-bffcd1e692c4 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:58.672434 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22a391-0c4c-4e8a-9e78-bffcd1e692c4 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:58.769229 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71d437-51ac-4456-ba9a-3059c82b7cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:58.770064 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71d437-51ac-4456-ba9a-3059c82b7cff admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:58.770352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71d437-51ac-4456-ba9a-3059c82b7cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:58.770626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71d437-51ac-4456-ba9a-3059c82b7cff admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:58.771317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71d437-51ac-4456-ba9a-3059c82b7cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:58.773717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71d437-51ac-4456-ba9a-3059c82b7cff admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:58.811263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 966/1927] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:07:58 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:07:58.815917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fceda-04c2-499d-9c53-0641146a91ae admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:58.830925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fceda-04c2-499d-9c53-0641146a91ae admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:58.843291 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0d473-a4c1-4e12-934d-f25115096069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:58.844307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0d473-a4c1-4e12-934d-f25115096069 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:58.844867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0d473-a4c1-4e12-934d-f25115096069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:58.845411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0d473-a4c1-4e12-934d-f25115096069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:58.846428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0d473-a4c1-4e12-934d-f25115096069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:58.851514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0d473-a4c1-4e12-934d-f25115096069 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:58.875328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 962/1928] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:58.878632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71d437-51ac-4456-ba9a-3059c82b7cff admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:58.894726 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71d437-51ac-4456-ba9a-3059c82b7cff admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:58.909119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82845dff-7965-4f68-accf-2a266ad03766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:58.909929 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845dff-7965-4f68-accf-2a266ad03766 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:58.910218 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845dff-7965-4f68-accf-2a266ad03766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:58.910693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845dff-7965-4f68-accf-2a266ad03766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:58.911421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82845dff-7965-4f68-accf-2a266ad03766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:58.916106 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82845dff-7965-4f68-accf-2a266ad03766 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:58.934027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 967/1929] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:07:58.938588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0d473-a4c1-4e12-934d-f25115096069 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:58.950817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0d473-a4c1-4e12-934d-f25115096069 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:58.965649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ea7a5-b7c7-43f4-a49f-a4be1adf3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:58.966374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ea7a5-b7c7-43f4-a49f-a4be1adf3748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:58.966723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ea7a5-b7c7-43f4-a49f-a4be1adf3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:58.968175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ea7a5-b7c7-43f4-a49f-a4be1adf3748 admin admin] PATH_INFO: `/v3/projects/471fe44b271e48dab321449ecfccb2a3/users/2eebc266138646acb04fd1cbab598b12/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:58.988279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ea7a5-b7c7-43f4-a49f-a4be1adf3748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=471fe44b271e48dab321449ecfccb2a3, user_id=2eebc266138646acb04fd1cbab598b12, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:58.991936 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ea7a5-b7c7-43f4-a49f-a4be1adf3748 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:59.023431 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 963/1930] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:07:58 2022] PUT /identity/v3/projects/471fe44b271e48dab321449ecfccb2a3/users/2eebc266138646acb04fd1cbab598b12/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:07:59.034899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510d8c9-5d83-4b17-a49a-5074d78f7d23 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:59.035387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510d8c9-5d83-4b17-a49a-5074d78f7d23 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:59.035727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510d8c9-5d83-4b17-a49a-5074d78f7d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:59.131839 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-4510d8c9-5d83-4b17-a49a-5074d78f7d23 None None] MFA Rules not processed for user `2eebc266138646acb04fd1cbab598b12`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:07:59.133184 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4510d8c9-5d83-4b17-a49a-5074d78f7d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:59.177878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:59.312563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 968/1931] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:07:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:07:59.325286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ea7a5-b7c7-43f4-a49f-a4be1adf3748 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:07:59.327466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-652ea7a5-b7c7-43f4-a49f-a4be1adf3748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:07:59.439366 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ea7a5-b7c7-43f4-a49f-a4be1adf3748 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:07:59.443911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f535c-079e-4c76-9e7d-576960ab4f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:07:59.444897 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f535c-079e-4c76-9e7d-576960ab4f85 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:07:59.445192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f535c-079e-4c76-9e7d-576960ab4f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:07:59.445469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f535c-079e-4c76-9e7d-576960ab4f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:07:59.451484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:07:59.457219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f535c-079e-4c76-9e7d-576960ab4f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:07:59.458873 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f535c-079e-4c76-9e7d-576960ab4f85 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:07:59.474357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 964/1932] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:07:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:03.387164 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510d8c9-5d83-4b17-a49a-5074d78f7d23 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:03.389533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4510d8c9-5d83-4b17-a49a-5074d78f7d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:03.474062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510d8c9-5d83-4b17-a49a-5074d78f7d23 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:03.478625 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:03.479279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:03.479554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:03.479823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] PATH_INFO: `/v3/users/3388875b82594451988646687c201091` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:03.481001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3388875b82594451988646687c201091)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:03.483746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:03.529914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3388875b82594451988646687c201091'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:03.530638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3388875b82594451988646687c201091'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:03.545805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3388875b82594451988646687c201091'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:03.552470 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3388875b82594451988646687c201091'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:03.553190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3388875b82594451988646687c201091'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:03.566334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 969/1933] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:08:03 2022] DELETE /identity/v3/users/3388875b82594451988646687c201091 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:03.570974 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cdafd-3b16-42d0-8b91-a99a1aaeba48 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:03.571528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cdafd-3b16-42d0-8b91-a99a1aaeba48 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:03.571528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cdafd-3b16-42d0-8b91-a99a1aaeba48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:03.590810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-c82cdafd-3b16-42d0-8b91-a99a1aaeba48 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:03.592813 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c82cdafd-3b16-42d0-8b91-a99a1aaeba48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:03.652140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 965/1934] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:03.663877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:03.673847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:03.721845 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:03.745828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d374f21-222b-474c-9819-9d07d9cbc078 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:03.749587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed6e1-d8cf-4b51-8489-1127b84a1715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:03.750216 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed6e1-d8cf-4b51-8489-1127b84a1715 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:03.750497 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed6e1-d8cf-4b51-8489-1127b84a1715 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:03.750767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed6e1-d8cf-4b51-8489-1127b84a1715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:03.760711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ed6e1-d8cf-4b51-8489-1127b84a1715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:03.762427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ed6e1-d8cf-4b51-8489-1127b84a1715 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:03.773405 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 970/1935] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:04.073801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cdafd-3b16-42d0-8b91-a99a1aaeba48 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:04.075270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c82cdafd-3b16-42d0-8b91-a99a1aaeba48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:04.087689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cdafd-3b16-42d0-8b91-a99a1aaeba48 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:04.092427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:04.093058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:04.093385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:04.093661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] PATH_INFO: `/v3/projects/79d35bd758904b888f67847522560758` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:04.094767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d35bd758904b888f67847522560758)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:04.096727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:04.147755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d35bd758904b888f67847522560758', 'request_id': 'req-eae41b7d-8288-471a-a659-1b259a94bf38'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.163068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d35bd758904b888f67847522560758', 'request_id': 'req-eae41b7d-8288-471a-a659-1b259a94bf38'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.163068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d35bd758904b888f67847522560758', 'request_id': 'req-eae41b7d-8288-471a-a659-1b259a94bf38'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.174725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] The token cache is being invalidate because project 79d35bd758904b888f67847522560758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:08:04.175047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.175808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.177227 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 966/1936] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:08:04 2022] DELETE /identity/v3/projects/79d35bd758904b888f67847522560758 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:04.181171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed6e1-d8cf-4b51-8489-1127b84a1715 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:04.192339 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed6e1-d8cf-4b51-8489-1127b84a1715 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:04.251272 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:04.251910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:04.252186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:04.252459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] PATH_INFO: `/v3/users/c36df8312ffd401e97a11780743a3dab` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:04.253579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c36df8312ffd401e97a11780743a3dab)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:04.255700 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:04.303109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c36df8312ffd401e97a11780743a3dab'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.304064 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c36df8312ffd401e97a11780743a3dab'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.316764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c36df8312ffd401e97a11780743a3dab'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.326455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c36df8312ffd401e97a11780743a3dab'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.327202 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c36df8312ffd401e97a11780743a3dab'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.348163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 971/1937] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:08:04 2022] DELETE /identity/v3/users/c36df8312ffd401e97a11780743a3dab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:04.701993 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:04.703498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:04.717463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae41b7d-8288-471a-a659-1b259a94bf38 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:04.775670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:04.776306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:04.776584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:04.776857 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] PATH_INFO: `/v3/projects/82aef298ec094370b2049c04dc9fbc53` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:04.778012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82aef298ec094370b2049c04dc9fbc53)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:04.780504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:04.801437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:04.827341 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82aef298ec094370b2049c04dc9fbc53', 'request_id': 'req-70958537-d3d6-4c17-a4c2-bd87460e7e98'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.833361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82aef298ec094370b2049c04dc9fbc53', 'request_id': 'req-70958537-d3d6-4c17-a4c2-bd87460e7e98'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.841113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82aef298ec094370b2049c04dc9fbc53', 'request_id': 'req-70958537-d3d6-4c17-a4c2-bd87460e7e98'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.851400 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] The token cache is being invalidate because project 82aef298ec094370b2049c04dc9fbc53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:08:04.851682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.852292 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.853759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 967/1938] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:08:04 2022] DELETE /identity/v3/projects/82aef298ec094370b2049c04dc9fbc53 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:04.857695 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:04.859206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:04.871742 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a3fcc-c0f3-44fa-a3d4-0b8953d3eadc admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:04.926569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:04.927219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:04.927499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:04.927778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] PATH_INFO: `/v3/users/1a1896f1ec6c44bfbe0754f003ad5cd9` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:04.928874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1896f1ec6c44bfbe0754f003ad5cd9)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:04.930774 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:04.933088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:04.977252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1896f1ec6c44bfbe0754f003ad5cd9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.977890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1896f1ec6c44bfbe0754f003ad5cd9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.989865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1896f1ec6c44bfbe0754f003ad5cd9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.996721 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1896f1ec6c44bfbe0754f003ad5cd9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:04.997366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ba9e9460-0513-4287-9014-c532c5c352fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1896f1ec6c44bfbe0754f003ad5cd9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:05.011885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 972/1939] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:08:04 2022] DELETE /identity/v3/users/1a1896f1ec6c44bfbe0754f003ad5cd9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:05.336239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:05.338241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:05.353072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70958537-d3d6-4c17-a4c2-bd87460e7e98 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:05.437613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:05.438258 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.438687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.438998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] PATH_INFO: `/v3/projects/9d2e838b0bdc4498b88941743ac20569` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.440156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d2e838b0bdc4498b88941743ac20569)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:05.442064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:05.492140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d2e838b0bdc4498b88941743ac20569', 'request_id': 'req-04649db5-598d-4ee2-b943-0028df163cd9'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:05.503635 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d2e838b0bdc4498b88941743ac20569', 'request_id': 'req-04649db5-598d-4ee2-b943-0028df163cd9'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:05.513300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d2e838b0bdc4498b88941743ac20569', 'request_id': 'req-04649db5-598d-4ee2-b943-0028df163cd9'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:05.524450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] The token cache is being invalidate because project 9d2e838b0bdc4498b88941743ac20569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:08:05.524734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:05.525357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-04649db5-598d-4ee2-b943-0028df163cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:05.526819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 968/1940] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:08:05 2022] DELETE /identity/v3/projects/9d2e838b0bdc4498b88941743ac20569 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:05.546044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1aff8-8a36-4846-9318-4b711701e04e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.546404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1aff8-8a36-4846-9318-4b711701e04e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.546685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1aff8-8a36-4846-9318-4b711701e04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.566969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6aa1aff8-8a36-4846-9318-4b711701e04e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:05.568160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6aa1aff8-8a36-4846-9318-4b711701e04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:05.627471 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 973/1941] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:05.635297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9817f7-52dd-464b-a3a4-0e7988e65de9 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.635575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9817f7-52dd-464b-a3a4-0e7988e65de9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.635848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9817f7-52dd-464b-a3a4-0e7988e65de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.656134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-fe9817f7-52dd-464b-a3a4-0e7988e65de9 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:05.657344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-fe9817f7-52dd-464b-a3a4-0e7988e65de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:05.669313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 969/1942] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:05.671086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1aff8-8a36-4846-9318-4b711701e04e None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:05.680599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1aff8-8a36-4846-9318-4b711701e04e None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:05.685145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95064b-ca13-473a-948f-6f56c11f6e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:05.685741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95064b-ca13-473a-948f-6f56c11f6e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.686016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95064b-ca13-473a-948f-6f56c11f6e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.686286 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95064b-ca13-473a-948f-6f56c11f6e1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.686925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95064b-ca13-473a-948f-6f56c11f6e1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:08:05.687256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95064b-ca13-473a-948f-6f56c11f6e1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:05.688840 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95064b-ca13-473a-948f-6f56c11f6e1a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:05.700891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 974/1943] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:08:05.705535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9817f7-52dd-464b-a3a4-0e7988e65de9 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:05.714633 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9817f7-52dd-464b-a3a4-0e7988e65de9 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:05.719818 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb629-1042-49db-8968-fbb275173cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:05.720423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb629-1042-49db-8968-fbb275173cea admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.720698 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb629-1042-49db-8968-fbb275173cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.720971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb629-1042-49db-8968-fbb275173cea admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.721633 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eb629-1042-49db-8968-fbb275173cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:05.723562 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eb629-1042-49db-8968-fbb275173cea admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:05.760546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 970/1944] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:08:05 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:08:05.762972 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95064b-ca13-473a-948f-6f56c11f6e1a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:05.772827 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95064b-ca13-473a-948f-6f56c11f6e1a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:05.828091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f430b41-5227-4d5c-b6f1-d0f4884dec37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:05.828683 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f430b41-5227-4d5c-b6f1-d0f4884dec37 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.828957 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f430b41-5227-4d5c-b6f1-d0f4884dec37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.829232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f430b41-5227-4d5c-b6f1-d0f4884dec37 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.829854 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f430b41-5227-4d5c-b6f1-d0f4884dec37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:05.831783 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f430b41-5227-4d5c-b6f1-d0f4884dec37 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:05.847635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:05.853905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 975/1945] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:08:05 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:08:05.857813 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb629-1042-49db-8968-fbb275173cea admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:05.868405 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb629-1042-49db-8968-fbb275173cea admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:05.873252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cc497-0423-4507-997e-f3080bec4290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:05.873855 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cc497-0423-4507-997e-f3080bec4290 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.874133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cc497-0423-4507-997e-f3080bec4290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.874430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cc497-0423-4507-997e-f3080bec4290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.875119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cc497-0423-4507-997e-f3080bec4290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:05.876715 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cc497-0423-4507-997e-f3080bec4290 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:05.878871 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:05.891241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 971/1946] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:08:05.892424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f430b41-5227-4d5c-b6f1-d0f4884dec37 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:05.902312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f430b41-5227-4d5c-b6f1-d0f4884dec37 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:05.907578 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514bb03-ad72-4a62-9982-5d0a9bfe7208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:05.908166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514bb03-ad72-4a62-9982-5d0a9bfe7208 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.908442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514bb03-ad72-4a62-9982-5d0a9bfe7208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.908711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514bb03-ad72-4a62-9982-5d0a9bfe7208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.909341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514bb03-ad72-4a62-9982-5d0a9bfe7208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:05.910931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514bb03-ad72-4a62-9982-5d0a9bfe7208 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:05.920825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 976/1947] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:08:05.924768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cc497-0423-4507-997e-f3080bec4290 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:05.934303 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cc497-0423-4507-997e-f3080bec4290 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:05.938878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f26b7-6123-46e1-ad1f-093b8c91f12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:05.939541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f26b7-6123-46e1-ad1f-093b8c91f12f admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.939816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f26b7-6123-46e1-ad1f-093b8c91f12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.940085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f26b7-6123-46e1-ad1f-093b8c91f12f admin admin] PATH_INFO: `/v3/projects/0121d8f74bf441afb820f567988c4d97/users/8c9a97e40b054a3fad1bd12e80c26708/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.954830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f26b7-6123-46e1-ad1f-093b8c91f12f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0121d8f74bf441afb820f567988c4d97, user_id=8c9a97e40b054a3fad1bd12e80c26708, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:05.957574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f26b7-6123-46e1-ad1f-093b8c91f12f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:05.970737 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 972/1948] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:08:05 2022] PUT /identity/v3/projects/0121d8f74bf441afb820f567988c4d97/users/8c9a97e40b054a3fad1bd12e80c26708/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:05.975735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6f892-2caf-45d0-9660-dfe1a5da6e55 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:05.976041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6f892-2caf-45d0-9660-dfe1a5da6e55 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:05.976292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6f892-2caf-45d0-9660-dfe1a5da6e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:05.996307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-b9e6f892-2caf-45d0-9660-dfe1a5da6e55 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:05.997501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b9e6f892-2caf-45d0-9660-dfe1a5da6e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:06.061766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 977/1949] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:08:06.115511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f26b7-6123-46e1-ad1f-093b8c91f12f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:06.118053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6f8f26b7-6123-46e1-ad1f-093b8c91f12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:06.215209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f26b7-6123-46e1-ad1f-093b8c91f12f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:06.219751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ad3f9-10f1-483f-99b7-b74d8833e7cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:06.220370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ad3f9-10f1-483f-99b7-b74d8833e7cf service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:06.221198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ad3f9-10f1-483f-99b7-b74d8833e7cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:06.221492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ad3f9-10f1-483f-99b7-b74d8833e7cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:06.232538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ad3f9-10f1-483f-99b7-b74d8833e7cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:06.234003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ad3f9-10f1-483f-99b7-b74d8833e7cf service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:06.245107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 973/1950] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:07.205733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1c197-48d9-4472-9ec6-57baf92bf14a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:07.206196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1c197-48d9-4472-9ec6-57baf92bf14a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:07.206429 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1c197-48d9-4472-9ec6-57baf92bf14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:07.234927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-8dc1c197-48d9-4472-9ec6-57baf92bf14a None None] MFA Rules not processed for user `8c9a97e40b054a3fad1bd12e80c26708`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:07.236201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8dc1c197-48d9-4472-9ec6-57baf92bf14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:07.301374 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 978/1951] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:07.313234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ad3f9-10f1-483f-99b7-b74d8833e7cf service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:07.314636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5f2ad3f9-10f1-483f-99b7-b74d8833e7cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:07.371653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ad3f9-10f1-483f-99b7-b74d8833e7cf service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:07.375842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d45803-65ab-4b71-945c-ffcfae5ac0c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:07.376450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d45803-65ab-4b71-945c-ffcfae5ac0c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:07.376724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d45803-65ab-4b71-945c-ffcfae5ac0c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:07.376994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d45803-65ab-4b71-945c-ffcfae5ac0c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:07.386402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d45803-65ab-4b71-945c-ffcfae5ac0c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:07.387856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d45803-65ab-4b71-945c-ffcfae5ac0c2 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:07.397009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 974/1952] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:10.867844 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312ca8c-1621-4097-808e-89dc30d347e9 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:10.868360 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312ca8c-1621-4097-808e-89dc30d347e9 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:10.868467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312ca8c-1621-4097-808e-89dc30d347e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:10.889290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-3312ca8c-1621-4097-808e-89dc30d347e9 None None] MFA Rules not processed for user `8c9a97e40b054a3fad1bd12e80c26708`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:10.890487 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3312ca8c-1621-4097-808e-89dc30d347e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:10.897402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 979/1953] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:11.706879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754c484-c3ea-46cd-b10e-54896e38d880 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:11.706879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754c484-c3ea-46cd-b10e-54896e38d880 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:11.707491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754c484-c3ea-46cd-b10e-54896e38d880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:11.776260 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b754c484-c3ea-46cd-b10e-54896e38d880 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:11.777613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b754c484-c3ea-46cd-b10e-54896e38d880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:11.886693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 975/1954] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:08:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:11.901538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312ca8c-1621-4097-808e-89dc30d347e9 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:11.902990 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3312ca8c-1621-4097-808e-89dc30d347e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:12.058711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312ca8c-1621-4097-808e-89dc30d347e9 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:12.082751 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523b0dd-d885-4e04-8bde-19cc2c52f80e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:12.083974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523b0dd-d885-4e04-8bde-19cc2c52f80e service swift] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:12.084312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523b0dd-d885-4e04-8bde-19cc2c52f80e service swift] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:12.084743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523b0dd-d885-4e04-8bde-19cc2c52f80e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:12.111393 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523b0dd-d885-4e04-8bde-19cc2c52f80e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:12.112992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523b0dd-d885-4e04-8bde-19cc2c52f80e service swift] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:12.141241 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:12.148945 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 980/1955] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:08:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:08:13.280081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754c484-c3ea-46cd-b10e-54896e38d880 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:13.281474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b754c484-c3ea-46cd-b10e-54896e38d880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:13.311067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754c484-c3ea-46cd-b10e-54896e38d880 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:13.326222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:13.326898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:13.327212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:13.327496 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] PATH_INFO: `/v3/users/2eebc266138646acb04fd1cbab598b12` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:13.329037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eebc266138646acb04fd1cbab598b12)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:13.331018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:13.366365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:13.411720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eebc266138646acb04fd1cbab598b12'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:13.412749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eebc266138646acb04fd1cbab598b12'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:13.428733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eebc266138646acb04fd1cbab598b12'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:13.442253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eebc266138646acb04fd1cbab598b12'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:13.455520 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eebc266138646acb04fd1cbab598b12'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:13.470381 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8082244-129b-4f7e-a9c5-922e7de05d83 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:13.470953 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8082244-129b-4f7e-a9c5-922e7de05d83 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:13.471095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8082244-129b-4f7e-a9c5-922e7de05d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:13.472566 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 976/1956] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:08:13 2022] DELETE /identity/v3/users/2eebc266138646acb04fd1cbab598b12 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:13.519191 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e8082244-129b-4f7e-a9c5-922e7de05d83 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:13.520667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e8082244-129b-4f7e-a9c5-922e7de05d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:13.616626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 981/1957] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:08:13.628458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:13.630526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:13.724759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582df06e-81d9-4fcf-8c03-bc6fca4475d5 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:13.732515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55290aa4-4488-415e-b863-be4cd17aa2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:13.733148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55290aa4-4488-415e-b863-be4cd17aa2fe service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:13.733424 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55290aa4-4488-415e-b863-be4cd17aa2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:13.739817 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55290aa4-4488-415e-b863-be4cd17aa2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:13.779990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55290aa4-4488-415e-b863-be4cd17aa2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:13.781454 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55290aa4-4488-415e-b863-be4cd17aa2fe service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:13.800976 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 977/1958] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:14.406551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8082244-129b-4f7e-a9c5-922e7de05d83 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:14.408394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e8082244-129b-4f7e-a9c5-922e7de05d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:14.420152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55290aa4-4488-415e-b863-be4cd17aa2fe service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:14.421762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-55290aa4-4488-415e-b863-be4cd17aa2fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:14.441025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8082244-129b-4f7e-a9c5-922e7de05d83 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:14.456327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55290aa4-4488-415e-b863-be4cd17aa2fe service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:14.471173 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:14.471838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:14.472124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:14.472404 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] PATH_INFO: `/v3/projects/471fe44b271e48dab321449ecfccb2a3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:14.478554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] RBAC: Authorizing `identity:delete_project(project_id=471fe44b271e48dab321449ecfccb2a3)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:14.478747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:14.480748 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:14.483856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:14.485335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:14.486964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] PATH_INFO: `/v3/users/a1ecc4efe96c4d39ad88bb3138caff12` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:14.490285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ecc4efe96c4d39ad88bb3138caff12)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:14.495812 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:14.639283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ecc4efe96c4d39ad88bb3138caff12'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.640923 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ecc4efe96c4d39ad88bb3138caff12'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.666944 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ecc4efe96c4d39ad88bb3138caff12'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.682849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '471fe44b271e48dab321449ecfccb2a3', 'request_id': 'req-98b9ed64-e656-4512-a3fd-bc335d2f10df'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.701858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '471fe44b271e48dab321449ecfccb2a3', 'request_id': 'req-98b9ed64-e656-4512-a3fd-bc335d2f10df'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.709471 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ecc4efe96c4d39ad88bb3138caff12'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.712800 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-87ad5693-738d-41bf-b2b0-a22ef562cbfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ecc4efe96c4d39ad88bb3138caff12'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.722454 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '471fe44b271e48dab321449ecfccb2a3', 'request_id': 'req-98b9ed64-e656-4512-a3fd-bc335d2f10df'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.752677 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 982/1959] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:08:14 2022] DELETE /identity/v3/users/a1ecc4efe96c4d39ad88bb3138caff12 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:14.761341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c110f0-4c9d-4844-9c5b-f9538ebf3f63 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:14.761873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c110f0-4c9d-4844-9c5b-f9538ebf3f63 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:14.762843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c110f0-4c9d-4844-9c5b-f9538ebf3f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:14.765207 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] The token cache is being invalidate because project 471fe44b271e48dab321449ecfccb2a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:08:14.768284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.774306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:14.777993 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 978/1960] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:08:14 2022] DELETE /identity/v3/projects/471fe44b271e48dab321449ecfccb2a3 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:08:14.818125 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-72c110f0-4c9d-4844-9c5b-f9538ebf3f63 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:14.822051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-72c110f0-4c9d-4844-9c5b-f9538ebf3f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:15.017905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 983/1961] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:15.027912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:15.029610 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:15.071609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:15.156375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ed64-e656-4512-a3fd-bc335d2f10df admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:15.274519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec909f-5513-49ae-842e-8b124c06ef2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:15.276343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec909f-5513-49ae-842e-8b124c06ef2e service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:15.277264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec909f-5513-49ae-842e-8b124c06ef2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:15.291716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec909f-5513-49ae-842e-8b124c06ef2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:15.313447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec909f-5513-49ae-842e-8b124c06ef2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:15.316344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec909f-5513-49ae-842e-8b124c06ef2e service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:15.334845 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 979/1962] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:16.261754 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c110f0-4c9d-4844-9c5b-f9538ebf3f63 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:16.277701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-72c110f0-4c9d-4844-9c5b-f9538ebf3f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:16.309224 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c110f0-4c9d-4844-9c5b-f9538ebf3f63 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:16.320709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:16.321513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:16.323521 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:16.323943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] PATH_INFO: `/v3/projects/ceb988fe6d614b6f860fb0a995a5444f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:16.331436 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb988fe6d614b6f860fb0a995a5444f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:16.334091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:16.366895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:16.459759 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb988fe6d614b6f860fb0a995a5444f', 'request_id': 'req-2bf29f11-b25f-4e2c-b883-641264cc7e96'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:16.477248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb988fe6d614b6f860fb0a995a5444f', 'request_id': 'req-2bf29f11-b25f-4e2c-b883-641264cc7e96'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:16.488096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb988fe6d614b6f860fb0a995a5444f', 'request_id': 'req-2bf29f11-b25f-4e2c-b883-641264cc7e96'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:16.509044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] The token cache is being invalidate because project ceb988fe6d614b6f860fb0a995a5444f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:08:16.509462 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:16.510412 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-2bf29f11-b25f-4e2c-b883-641264cc7e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:16.513161 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 984/1963] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:08:16 2022] DELETE /identity/v3/projects/ceb988fe6d614b6f860fb0a995a5444f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:16.519444 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78897b-8843-48e9-bd19-1d78159c83bd None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:16.520037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78897b-8843-48e9-bd19-1d78159c83bd None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:16.520037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78897b-8843-48e9-bd19-1d78159c83bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:16.584947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-1e78897b-8843-48e9-bd19-1d78159c83bd None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:16.587741 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1e78897b-8843-48e9-bd19-1d78159c83bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:16.722621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 980/1964] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:16.736907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85100b-3028-43e8-b0b8-942ecafdd6f5 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:16.737199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85100b-3028-43e8-b0b8-942ecafdd6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:16.737484 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85100b-3028-43e8-b0b8-942ecafdd6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:16.803287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-5f85100b-3028-43e8-b0b8-942ecafdd6f5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:16.804760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-5f85100b-3028-43e8-b0b8-942ecafdd6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:16.824261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 985/1965] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:16.825385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78897b-8843-48e9-bd19-1d78159c83bd None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:16.843359 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78897b-8843-48e9-bd19-1d78159c83bd None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:16.858344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdec191-cfcd-4354-97a8-5ce75c4a4b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:16.861570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdec191-cfcd-4354-97a8-5ce75c4a4b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:16.861570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdec191-cfcd-4354-97a8-5ce75c4a4b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:16.861570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdec191-cfcd-4354-97a8-5ce75c4a4b18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:16.861570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdec191-cfcd-4354-97a8-5ce75c4a4b18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:08:16.861570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdec191-cfcd-4354-97a8-5ce75c4a4b18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:16.863595 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdec191-cfcd-4354-97a8-5ce75c4a4b18 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:16.887599 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 981/1966] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:08:16.891315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85100b-3028-43e8-b0b8-942ecafdd6f5 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:16.908654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85100b-3028-43e8-b0b8-942ecafdd6f5 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:16.927348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9b04f-9b6c-4e8f-8e31-8774663509cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:16.928018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9b04f-9b6c-4e8f-8e31-8774663509cc admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:16.928284 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9b04f-9b6c-4e8f-8e31-8774663509cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:16.928585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9b04f-9b6c-4e8f-8e31-8774663509cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:16.931559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9b04f-9b6c-4e8f-8e31-8774663509cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:16.931559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9b04f-9b6c-4e8f-8e31-8774663509cc admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:17.010616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 986/1967] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:08:16 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:08:17.015110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdec191-cfcd-4354-97a8-5ce75c4a4b18 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:17.031842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdec191-cfcd-4354-97a8-5ce75c4a4b18 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:17.177133 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132e651-0750-488a-9f3f-f3a638b22093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:17.180679 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e651-0750-488a-9f3f-f3a638b22093 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:17.181027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e651-0750-488a-9f3f-f3a638b22093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:17.181314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e651-0750-488a-9f3f-f3a638b22093 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:17.182154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132e651-0750-488a-9f3f-f3a638b22093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:17.187521 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132e651-0750-488a-9f3f-f3a638b22093 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:17.232460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:17.276406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 982/1968] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:08:17 2022] POST /identity/v3/users => generated 287 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:08:17.278235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9b04f-9b6c-4e8f-8e31-8774663509cc admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:17.292084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9b04f-9b6c-4e8f-8e31-8774663509cc admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:17.298426 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66abd31-879d-44a4-bf56-f653be243f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:17.300154 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66abd31-879d-44a4-bf56-f653be243f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:17.300723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66abd31-879d-44a4-bf56-f653be243f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:17.301243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66abd31-879d-44a4-bf56-f653be243f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:17.302302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66abd31-879d-44a4-bf56-f653be243f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:17.305643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66abd31-879d-44a4-bf56-f653be243f5e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:17.325779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132e651-0750-488a-9f3f-f3a638b22093 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:17.337153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 987/1969] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:08:17.338920 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132e651-0750-488a-9f3f-f3a638b22093 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:17.344388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665af43-9b09-4321-b2bc-7b9595bc771a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:17.345155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665af43-9b09-4321-b2bc-7b9595bc771a admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:17.345505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665af43-9b09-4321-b2bc-7b9595bc771a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:17.346030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665af43-9b09-4321-b2bc-7b9595bc771a admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:17.346924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665af43-9b09-4321-b2bc-7b9595bc771a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:17.349187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665af43-9b09-4321-b2bc-7b9595bc771a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:17.367269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66abd31-879d-44a4-bf56-f653be243f5e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:17.375550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 983/1970] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:08:17.380316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66abd31-879d-44a4-bf56-f653be243f5e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:17.388154 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e40a82-94ed-45d6-a6e3-dc6556595b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:17.389160 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e40a82-94ed-45d6-a6e3-dc6556595b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:17.389799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e40a82-94ed-45d6-a6e3-dc6556595b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:17.390323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e40a82-94ed-45d6-a6e3-dc6556595b1f admin admin] PATH_INFO: `/v3/projects/1e000f0a6ba34f6e90895dfa943e0f9d/users/6ef01b0dd5134adc89a15b94aa614cfc/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:17.410235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e40a82-94ed-45d6-a6e3-dc6556595b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e000f0a6ba34f6e90895dfa943e0f9d, user_id=6ef01b0dd5134adc89a15b94aa614cfc, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:17.415496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e40a82-94ed-45d6-a6e3-dc6556595b1f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:17.481382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a5294-7741-44bf-a701-bb2f08522122 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:17.481608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a5294-7741-44bf-a701-bb2f08522122 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:17.483093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a5294-7741-44bf-a701-bb2f08522122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:17.486155 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 988/1971] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:08:17 2022] PUT /identity/v3/projects/1e000f0a6ba34f6e90895dfa943e0f9d/users/6ef01b0dd5134adc89a15b94aa614cfc/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:17.572414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-1e4a5294-7741-44bf-a701-bb2f08522122 None None] MFA Rules not processed for user `6ef01b0dd5134adc89a15b94aa614cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:17.573664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1e4a5294-7741-44bf-a701-bb2f08522122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:17.719783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 984/1972] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:17.744249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e40a82-94ed-45d6-a6e3-dc6556595b1f admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:17.748451 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-27e40a82-94ed-45d6-a6e3-dc6556595b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:17.848792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e40a82-94ed-45d6-a6e3-dc6556595b1f admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:17.852556 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58234f8-1fd7-48be-9f23-a6351e1758f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:17.853165 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58234f8-1fd7-48be-9f23-a6351e1758f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:17.853747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58234f8-1fd7-48be-9f23-a6351e1758f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:17.853747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58234f8-1fd7-48be-9f23-a6351e1758f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:17.863915 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58234f8-1fd7-48be-9f23-a6351e1758f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:17.865546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58234f8-1fd7-48be-9f23-a6351e1758f2 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:17.876500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 989/1973] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:17.915176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a5294-7741-44bf-a701-bb2f08522122 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:17.995406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a5294-7741-44bf-a701-bb2f08522122 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:18.002212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:18.003022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:18.003223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:18.003662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] PATH_INFO: `/v3/users/6ef01b0dd5134adc89a15b94aa614cfc` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:18.005466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef01b0dd5134adc89a15b94aa614cfc)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:18.011552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:18.086044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef01b0dd5134adc89a15b94aa614cfc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:18.086886 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef01b0dd5134adc89a15b94aa614cfc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:18.114186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef01b0dd5134adc89a15b94aa614cfc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:18.122795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef01b0dd5134adc89a15b94aa614cfc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:18.126494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:18.134644 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef01b0dd5134adc89a15b94aa614cfc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:18.145671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 985/1974] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:08:17 2022] DELETE /identity/v3/users/6ef01b0dd5134adc89a15b94aa614cfc => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:18.150233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7b783-11a2-4cc3-9b88-b783249020eb None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:18.150520 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7b783-11a2-4cc3-9b88-b783249020eb None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:18.150843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7b783-11a2-4cc3-9b88-b783249020eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:18.176037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e0e7b783-11a2-4cc3-9b88-b783249020eb None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:18.177612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e0e7b783-11a2-4cc3-9b88-b783249020eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:18.220948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:18.276267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 990/1975] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:18.285144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:18.286895 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:18.395179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a634f5-21e0-4fa8-8f85-af09773ee3d1 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:18.400242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883b924-f7d9-4d94-a024-bf3acd266252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:18.401172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883b924-f7d9-4d94-a024-bf3acd266252 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:18.401670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883b924-f7d9-4d94-a024-bf3acd266252 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:18.402148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883b924-f7d9-4d94-a024-bf3acd266252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:18.415158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883b924-f7d9-4d94-a024-bf3acd266252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:18.416827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883b924-f7d9-4d94-a024-bf3acd266252 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:18.429268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 986/1976] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:19.126587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7b783-11a2-4cc3-9b88-b783249020eb None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:19.128032 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e0e7b783-11a2-4cc3-9b88-b783249020eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:19.152823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7b783-11a2-4cc3-9b88-b783249020eb None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:19.165207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:19.165938 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:19.166321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:19.172968 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] PATH_INFO: `/v3/projects/1e000f0a6ba34f6e90895dfa943e0f9d` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:19.175741 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e000f0a6ba34f6e90895dfa943e0f9d)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:19.179558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:19.301444 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e000f0a6ba34f6e90895dfa943e0f9d', 'request_id': 'req-00085774-b942-4fd5-880e-10754d0f5906'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:19.313600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e000f0a6ba34f6e90895dfa943e0f9d', 'request_id': 'req-00085774-b942-4fd5-880e-10754d0f5906'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:19.321453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e000f0a6ba34f6e90895dfa943e0f9d', 'request_id': 'req-00085774-b942-4fd5-880e-10754d0f5906'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:19.337079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] The token cache is being invalidate because project 1e000f0a6ba34f6e90895dfa943e0f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:08:19.337352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:19.338010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-00085774-b942-4fd5-880e-10754d0f5906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:19.339598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 991/1977] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:08:19 2022] DELETE /identity/v3/projects/1e000f0a6ba34f6e90895dfa943e0f9d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:28.423537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=28 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:29.333659 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:34.334217 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea68b-ad77-49ba-9a72-3b0c4495ea9c None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:34.334752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea68b-ad77-49ba-9a72-3b0c4495ea9c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:34.334849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea68b-ad77-49ba-9a72-3b0c4495ea9c None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:34.336197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 987/1978] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:08:34.342814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd1044-a925-4c78-aae2-30c530f0aa7c None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:34.343286 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd1044-a925-4c78-aae2-30c530f0aa7c None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:34.343519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd1044-a925-4c78-aae2-30c530f0aa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:34.363506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-0bfd1044-a925-4c78-aae2-30c530f0aa7c None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:34.364817 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0bfd1044-a925-4c78-aae2-30c530f0aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:34.435300 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 992/1979] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:08:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:34.451492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafea68b-ad77-49ba-9a72-3b0c4495ea9c None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:34.453179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-dafea68b-ad77-49ba-9a72-3b0c4495ea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:34.571958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafea68b-ad77-49ba-9a72-3b0c4495ea9c None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:34.576154 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041e141-f38b-45d1-a159-858094837b22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8ec16bd9254b23bfd2b60551af9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:34.576819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041e141-f38b-45d1-a159-858094837b22 service glance] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:34.577269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041e141-f38b-45d1-a159-858094837b22 service glance] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:34.577780 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041e141-f38b-45d1-a159-858094837b22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:34.589571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041e141-f38b-45d1-a159-858094837b22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:34.591330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041e141-f38b-45d1-a159-858094837b22 service glance] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:34.605237 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 988/1980] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:34.686613 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01ccc-9f2f-4c78-89bb-87cd7787052d None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:34.687036 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01ccc-9f2f-4c78-89bb-87cd7787052d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:34.687431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01ccc-9f2f-4c78-89bb-87cd7787052d None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:34.689003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 993/1981] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:08:34.694716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6595a-897f-4f9a-b660-34a6a2319b75 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:34.695234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6595a-897f-4f9a-b660-34a6a2319b75 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:34.695812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6595a-897f-4f9a-b660-34a6a2319b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:34.716947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-49d6595a-897f-4f9a-b660-34a6a2319b75 None None] MFA Rules not processed for user `9cffb662f4104e349c81c578bf66ab7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:34.718132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-49d6595a-897f-4f9a-b660-34a6a2319b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:34.788628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 989/1982] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:08:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:34.803232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01ccc-9f2f-4c78-89bb-87cd7787052d None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:34.805254 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-84b01ccc-9f2f-4c78-89bb-87cd7787052d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:34.819442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01ccc-9f2f-4c78-89bb-87cd7787052d None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:34.824993 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e347bd-a7e4-48ee-8788-9e31741efef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:34.825678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e347bd-a7e4-48ee-8788-9e31741efef4 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:34.826619 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e347bd-a7e4-48ee-8788-9e31741efef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:34.826893 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e347bd-a7e4-48ee-8788-9e31741efef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:34.836491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e347bd-a7e4-48ee-8788-9e31741efef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:34.839212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e347bd-a7e4-48ee-8788-9e31741efef4 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:34.848591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 994/1983] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:34.936721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6167b5-ba8d-4673-bf4b-e1b3befabbab None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:34.937179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6167b5-ba8d-4673-bf4b-e1b3befabbab None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:34.937451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6167b5-ba8d-4673-bf4b-e1b3befabbab None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:34.938777 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 990/1984] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:08:34.944382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff066-8783-442c-8a51-82af41305791 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:34.944764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff066-8783-442c-8a51-82af41305791 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:34.945130 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff066-8783-442c-8a51-82af41305791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:34.966838 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-3a3ff066-8783-442c-8a51-82af41305791 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:34.968095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3a3ff066-8783-442c-8a51-82af41305791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:34.977566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 995/1985] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:08:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:34.989215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6167b5-ba8d-4673-bf4b-e1b3befabbab None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:35.000981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6167b5-ba8d-4673-bf4b-e1b3befabbab None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:35.004513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279a587-f999-4706-a905-142210bde81b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:35.005105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a587-f999-4706-a905-142210bde81b service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:35.005377 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a587-f999-4706-a905-142210bde81b service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:35.005639 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a587-f999-4706-a905-142210bde81b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:35.016340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279a587-f999-4706-a905-142210bde81b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:35.017802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279a587-f999-4706-a905-142210bde81b service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:35.030632 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 991/1986] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:35.293248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff066-8783-442c-8a51-82af41305791 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:35.303274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff066-8783-442c-8a51-82af41305791 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:35.307078 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:35.307686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:35.308145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:35.308416 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:35.310475 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:35.391394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:35.393572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:35.405651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 996/1987] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:08:35.676924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ff5b-f62c-4f93-809e-e2ea6b20255f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:35.677349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ff5b-f62c-4f93-809e-e2ea6b20255f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:35.677458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ff5b-f62c-4f93-809e-e2ea6b20255f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:35.707994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-69b9ff5b-f62c-4f93-809e-e2ea6b20255f None None] MFA Rules not processed for user `8c9a97e40b054a3fad1bd12e80c26708`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:35.709210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-69b9ff5b-f62c-4f93-809e-e2ea6b20255f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:35.779022 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 992/1988] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:41.488700 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:41.490567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:41.568495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ac0a1-6400-4dce-81e4-22fea6b8116f service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:41.573136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:41.573771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:41.574046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:41.574345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] PATH_INFO: `/v3/users/fa16c5a0cd154c45938aff7a0ce987f1` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:41.575631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa16c5a0cd154c45938aff7a0ce987f1)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:41.577756 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:41.586696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=100 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:41.644736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa16c5a0cd154c45938aff7a0ce987f1'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:41.645481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa16c5a0cd154c45938aff7a0ce987f1'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:41.659563 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa16c5a0cd154c45938aff7a0ce987f1'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:41.667096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa16c5a0cd154c45938aff7a0ce987f1'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:41.668497 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa16c5a0cd154c45938aff7a0ce987f1'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:41.686240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 997/1989] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:08:41 2022] DELETE /identity/v3/users/fa16c5a0cd154c45938aff7a0ce987f1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:08:41.691045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375089f-e1aa-45f2-bb73-7e00d6c32652 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:41.691597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375089f-e1aa-45f2-bb73-7e00d6c32652 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:41.691597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375089f-e1aa-45f2-bb73-7e00d6c32652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:41.711374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-e375089f-e1aa-45f2-bb73-7e00d6c32652 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:41.712727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e375089f-e1aa-45f2-bb73-7e00d6c32652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:41.766690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:41.786832 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 993/1990] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:41.797377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:41.798885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:41.858368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d68f4-54fa-49ac-9023-ec9c8730796b admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:41.861994 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfa172-c2af-459e-ba0f-b5b4466a3e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:41.862609 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfa172-c2af-459e-ba0f-b5b4466a3e03 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:41.862884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfa172-c2af-459e-ba0f-b5b4466a3e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:41.863177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfa172-c2af-459e-ba0f-b5b4466a3e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:41.872701 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfa172-c2af-459e-ba0f-b5b4466a3e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:41.874138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfa172-c2af-459e-ba0f-b5b4466a3e03 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:41.884371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 998/1991] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:08:42.162734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375089f-e1aa-45f2-bb73-7e00d6c32652 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:42.164899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e375089f-e1aa-45f2-bb73-7e00d6c32652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:42.182250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375089f-e1aa-45f2-bb73-7e00d6c32652 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:42.191952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:42.192745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:42.193179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:42.193516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] PATH_INFO: `/v3/projects/d802b8ce56e046d6af6058d14b5e1ac1` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:42.195106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d802b8ce56e046d6af6058d14b5e1ac1)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:42.197314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:42.264828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd802b8ce56e046d6af6058d14b5e1ac1', 'request_id': 'req-33749b1d-5414-4f45-8d35-23d8e41cd722'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:42.275026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd802b8ce56e046d6af6058d14b5e1ac1', 'request_id': 'req-33749b1d-5414-4f45-8d35-23d8e41cd722'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:42.308005 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd802b8ce56e046d6af6058d14b5e1ac1', 'request_id': 'req-33749b1d-5414-4f45-8d35-23d8e41cd722'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:42.357453 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] The token cache is being invalidate because project d802b8ce56e046d6af6058d14b5e1ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:08:42.359255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:42.361771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-33749b1d-5414-4f45-8d35-23d8e41cd722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:08:42.364344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 994/1992] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:08:42 2022] DELETE /identity/v3/projects/d802b8ce56e046d6af6058d14b5e1ac1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:08:42.467326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64729927-32c3-4455-97a4-ffbe0d48cf99 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:42.467609 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64729927-32c3-4455-97a4-ffbe0d48cf99 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:42.468105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64729927-32c3-4455-97a4-ffbe0d48cf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:42.511969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-64729927-32c3-4455-97a4-ffbe0d48cf99 None None] MFA Rules not processed for user `c8ca34fc5c5743d0b5e3f4f97ae285e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:42.515472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-64729927-32c3-4455-97a4-ffbe0d48cf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:42.654120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 999/1993] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:08:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:45.947596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68b895-575f-40c1-9eea-aa786f3754c9 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:45.948092 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68b895-575f-40c1-9eea-aa786f3754c9 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:45.948299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68b895-575f-40c1-9eea-aa786f3754c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:45.967924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-8d68b895-575f-40c1-9eea-aa786f3754c9 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:08:45.969394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8d68b895-575f-40c1-9eea-aa786f3754c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:46.026218 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 995/1994] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:08:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:08:46.048842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64729927-32c3-4455-97a4-ffbe0d48cf99 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:08:46.050315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-64729927-32c3-4455-97a4-ffbe0d48cf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:08:46.097885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:46.141279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64729927-32c3-4455-97a4-ffbe0d48cf99 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:08:46.146706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4700b-0b92-48fd-810a-dcc60868d738 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:08:46.147710 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4700b-0b92-48fd-810a-dcc60868d738 service swift] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:08:46.147945 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4700b-0b92-48fd-810a-dcc60868d738 service swift] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:08:46.148261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4700b-0b92-48fd-810a-dcc60868d738 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:08:46.157478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4700b-0b92-48fd-810a-dcc60868d738 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:08:46.159098 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4700b-0b92-48fd-810a-dcc60868d738 service swift] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:08:46.168524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1000/1995] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:08:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:08:56.019483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:08:56.164179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=17 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:09:13.139786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880eeba-d7ac-45bf-a4d9-4c887d9c8e93 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:13.140323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880eeba-d7ac-45bf-a4d9-4c887d9c8e93 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:13.140707 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880eeba-d7ac-45bf-a4d9-4c887d9c8e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:13.175301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4880eeba-d7ac-45bf-a4d9-4c887d9c8e93 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:09:13.176630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4880eeba-d7ac-45bf-a4d9-4c887d9c8e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:09:13.185537 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 996/1996] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:09:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:09:13.197352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4700b-0b92-48fd-810a-dcc60868d738 service swift] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:13.208528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4700b-0b92-48fd-810a-dcc60868d738 service swift] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:13.213865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3037f-edee-42a3-a73c-e2910d3bd58b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:13.214580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3037f-edee-42a3-a73c-e2910d3bd58b service swift] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:13.214958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3037f-edee-42a3-a73c-e2910d3bd58b service swift] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:13.216147 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3037f-edee-42a3-a73c-e2910d3bd58b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:13.224427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3037f-edee-42a3-a73c-e2910d3bd58b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:13.226133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3037f-edee-42a3-a73c-e2910d3bd58b service swift] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:13.237810 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1001/1997] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:09:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:09:13.643030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28029c6b-c2d7-4bfd-946d-e45995e30ae5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:13.643655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28029c6b-c2d7-4bfd-946d-e45995e30ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:13.644106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28029c6b-c2d7-4bfd-946d-e45995e30ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:13.667172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-28029c6b-c2d7-4bfd-946d-e45995e30ae5 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:09:13.668893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-28029c6b-c2d7-4bfd-946d-e45995e30ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:09:13.677297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 997/1998] 10.222.0.58 () {58 vars in 967 bytes} [Fri Oct 28 19:09:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:09:13.697878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3037f-edee-42a3-a73c-e2910d3bd58b service swift] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:13.712600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3037f-edee-42a3-a73c-e2910d3bd58b service swift] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:13.719270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41080ccd-6634-46f5-bb83-6db69d10e75a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:13.720132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080ccd-6634-46f5-bb83-6db69d10e75a service swift] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:13.720601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080ccd-6634-46f5-bb83-6db69d10e75a service swift] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:13.721076 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080ccd-6634-46f5-bb83-6db69d10e75a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:13.731535 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41080ccd-6634-46f5-bb83-6db69d10e75a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:13.735794 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41080ccd-6634-46f5-bb83-6db69d10e75a service swift] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:13.745504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1002/1999] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:09:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 28 19:09:23.660524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=6 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:09:23.741422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=12 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:09:28.776802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13be229-bec9-44fe-9498-4dbcedf5c120 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:28.777234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13be229-bec9-44fe-9498-4dbcedf5c120 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:28.777350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13be229-bec9-44fe-9498-4dbcedf5c120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:28.796646 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d13be229-bec9-44fe-9498-4dbcedf5c120 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:09:28.797832 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d13be229-bec9-44fe-9498-4dbcedf5c120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:09:28.804957 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 998/2000] 10.222.0.58 () {58 vars in 967 bytes} [Fri Oct 28 19:09:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:09:28.817383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41080ccd-6634-46f5-bb83-6db69d10e75a service swift] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:28.828345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41080ccd-6634-46f5-bb83-6db69d10e75a service swift] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:28.834497 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa2f15-995c-45eb-bb00-400feaa686ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:28.835725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa2f15-995c-45eb-bb00-400feaa686ee service swift] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:28.836188 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa2f15-995c-45eb-bb00-400feaa686ee service swift] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:28.836639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa2f15-995c-45eb-bb00-400feaa686ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:28.845896 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa2f15-995c-45eb-bb00-400feaa686ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:28.847497 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa2f15-995c-45eb-bb00-400feaa686ee service swift] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:28.856445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1003/2001] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:09:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:09:30.769985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13be229-bec9-44fe-9498-4dbcedf5c120 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:30.771518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d13be229-bec9-44fe-9498-4dbcedf5c120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:09:30.836086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13be229-bec9-44fe-9498-4dbcedf5c120 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:30.841049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:30.841673 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:30.841946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:30.842216 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] PATH_INFO: `/v3/users/8c9a97e40b054a3fad1bd12e80c26708` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:30.843372 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c9a97e40b054a3fad1bd12e80c26708)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:30.845293 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:30.889985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c9a97e40b054a3fad1bd12e80c26708'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:30.890625 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c9a97e40b054a3fad1bd12e80c26708'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:30.902825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c9a97e40b054a3fad1bd12e80c26708'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:30.909861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c9a97e40b054a3fad1bd12e80c26708'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:30.917487 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d8990859-3d8f-4a45-9f15-a544f0b549e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c9a97e40b054a3fad1bd12e80c26708'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:30.923425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 999/2002] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:09:30 2022] DELETE /identity/v3/users/8c9a97e40b054a3fad1bd12e80c26708 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:09:31.392948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa2f15-995c-45eb-bb00-400feaa686ee service swift] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:31.406718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa2f15-995c-45eb-bb00-400feaa686ee service swift] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:31.464942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:31.465530 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.465806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.466075 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] PATH_INFO: `/v3/projects/0121d8f74bf441afb820f567988c4d97` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:31.467246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0121d8f74bf441afb820f567988c4d97)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:31.469144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:31.516085 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0121d8f74bf441afb820f567988c4d97', 'request_id': 'req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:31.523352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0121d8f74bf441afb820f567988c4d97', 'request_id': 'req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:31.529735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0121d8f74bf441afb820f567988c4d97', 'request_id': 'req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:31.539907 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] The token cache is being invalidate because project 0121d8f74bf441afb820f567988c4d97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:09:31.540196 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:31.541348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-09ea26ea-60b1-4962-b32b-960a2fdfeb2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:09:31.542829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1004/2003] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:09:31 2022] DELETE /identity/v3/projects/0121d8f74bf441afb820f567988c4d97 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:09:31.547689 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb84de8-1c9a-4dc4-b491-8b94b431639c None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.547975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb84de8-1c9a-4dc4-b491-8b94b431639c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.548246 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb84de8-1c9a-4dc4-b491-8b94b431639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:31.568309 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6fb84de8-1c9a-4dc4-b491-8b94b431639c None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:09:31.569523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6fb84de8-1c9a-4dc4-b491-8b94b431639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:09:31.628999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1000/2004] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:09:31.638175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be149802-cf23-4adc-aced-c8670e756aa4 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.638464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be149802-cf23-4adc-aced-c8670e756aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.638739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be149802-cf23-4adc-aced-c8670e756aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:31.658724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-be149802-cf23-4adc-aced-c8670e756aa4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:09:31.659985 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-be149802-cf23-4adc-aced-c8670e756aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:09:31.667763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1005/2005] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:09:31.671969 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb84de8-1c9a-4dc4-b491-8b94b431639c None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:31.681467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb84de8-1c9a-4dc4-b491-8b94b431639c None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:31.686002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa83665-ca70-4bdd-a1d2-ad73c5a6c994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:31.686588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa83665-ca70-4bdd-a1d2-ad73c5a6c994 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.686861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa83665-ca70-4bdd-a1d2-ad73c5a6c994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.687155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa83665-ca70-4bdd-a1d2-ad73c5a6c994 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:31.687786 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa83665-ca70-4bdd-a1d2-ad73c5a6c994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:09:31.688083 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa83665-ca70-4bdd-a1d2-ad73c5a6c994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:31.689664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa83665-ca70-4bdd-a1d2-ad73c5a6c994 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:31.698561 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:09:31.702337 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1001/2006] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:09:31.706635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be149802-cf23-4adc-aced-c8670e756aa4 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:31.716379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be149802-cf23-4adc-aced-c8670e756aa4 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:31.720908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a14213-f9ef-49e0-aed4-0af2dacfee4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:31.721505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a14213-f9ef-49e0-aed4-0af2dacfee4f admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.721779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a14213-f9ef-49e0-aed4-0af2dacfee4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.722043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a14213-f9ef-49e0-aed4-0af2dacfee4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:31.722667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a14213-f9ef-49e0-aed4-0af2dacfee4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:31.724694 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a14213-f9ef-49e0-aed4-0af2dacfee4f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:31.755796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1006/2007] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:09:31 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:09:31.756930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa83665-ca70-4bdd-a1d2-ad73c5a6c994 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:31.767871 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa83665-ca70-4bdd-a1d2-ad73c5a6c994 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:31.823425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc68506-e817-46f2-a805-041e6a22240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:31.824085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68506-e817-46f2-a805-041e6a22240c admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.824287 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68506-e817-46f2-a805-041e6a22240c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.824555 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68506-e817-46f2-a805-041e6a22240c admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:31.825171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc68506-e817-46f2-a805-041e6a22240c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:31.827124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc68506-e817-46f2-a805-041e6a22240c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:31.846727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1002/2008] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:09:31 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:09:31.850902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a14213-f9ef-49e0-aed4-0af2dacfee4f admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:31.861172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a14213-f9ef-49e0-aed4-0af2dacfee4f admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:31.865686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad8885-2108-49d9-b908-25e5d72cf00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:31.866277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad8885-2108-49d9-b908-25e5d72cf00b admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.866581 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad8885-2108-49d9-b908-25e5d72cf00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.866845 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad8885-2108-49d9-b908-25e5d72cf00b admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:31.867508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad8885-2108-49d9-b908-25e5d72cf00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:31.869670 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad8885-2108-49d9-b908-25e5d72cf00b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:31.879383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1007/2009] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:09:31.883089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc68506-e817-46f2-a805-041e6a22240c admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:31.893664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc68506-e817-46f2-a805-041e6a22240c admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:31.898205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d9276-529c-4e36-a051-aee4b0cb5290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:31.898784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9276-529c-4e36-a051-aee4b0cb5290 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.899099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9276-529c-4e36-a051-aee4b0cb5290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.899374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9276-529c-4e36-a051-aee4b0cb5290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:31.900045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d9276-529c-4e36-a051-aee4b0cb5290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:31.901621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d9276-529c-4e36-a051-aee4b0cb5290 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:31.911991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1003/2010] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:09:31.915787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad8885-2108-49d9-b908-25e5d72cf00b admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:31.925775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad8885-2108-49d9-b908-25e5d72cf00b admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:31.930289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a53bd-7461-45da-bd2e-9e164944e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:31.930925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a53bd-7461-45da-bd2e-9e164944e2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.931770 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a53bd-7461-45da-bd2e-9e164944e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.932041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a53bd-7461-45da-bd2e-9e164944e2dd admin admin] PATH_INFO: `/v3/projects/4dada98c0d7a47e3af3f20cb633ddebc/users/f5820d3d7c5c46938dcf72bb05938eeb/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:31.946095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a53bd-7461-45da-bd2e-9e164944e2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dada98c0d7a47e3af3f20cb633ddebc, user_id=f5820d3d7c5c46938dcf72bb05938eeb, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:31.949341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a53bd-7461-45da-bd2e-9e164944e2dd admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:31.961870 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1008/2011] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:09:31 2022] PUT /identity/v3/projects/4dada98c0d7a47e3af3f20cb633ddebc/users/f5820d3d7c5c46938dcf72bb05938eeb/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:09:31.971905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e5f9e-f6dc-4a7e-952a-8e75b5a9c1bf None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:31.972149 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e5f9e-f6dc-4a7e-952a-8e75b5a9c1bf None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:31.972420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e5f9e-f6dc-4a7e-952a-8e75b5a9c1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:32.001526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f65e5f9e-f6dc-4a7e-952a-8e75b5a9c1bf None None] MFA Rules not processed for user `f5820d3d7c5c46938dcf72bb05938eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:09:32.002737 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f65e5f9e-f6dc-4a7e-952a-8e75b5a9c1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:09:32.065699 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1004/2012] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:09:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:09:32.078110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a53bd-7461-45da-bd2e-9e164944e2dd admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:32.079851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-920a53bd-7461-45da-bd2e-9e164944e2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:09:32.093942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:09:32.137402 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a53bd-7461-45da-bd2e-9e164944e2dd admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:32.140986 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed6c9c-76e2-4d54-afcd-55a4db318037 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8ec16bd9254b23bfd2b60551af9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:32.141587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed6c9c-76e2-4d54-afcd-55a4db318037 service glance] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:32.141862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed6c9c-76e2-4d54-afcd-55a4db318037 service glance] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:32.142130 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed6c9c-76e2-4d54-afcd-55a4db318037 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:32.150193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed6c9c-76e2-4d54-afcd-55a4db318037 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:32.151638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed6c9c-76e2-4d54-afcd-55a4db318037 service glance] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:09:32.160146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1009/2013] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:09:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:09:42.062791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:09:42.156295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=20 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:09:59.922718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e5f9e-f6dc-4a7e-952a-8e75b5a9c1bf None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:09:59.983099 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e5f9e-f6dc-4a7e-952a-8e75b5a9c1bf None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:09:59.988027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:09:59.988647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:09:59.988925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:09:59.989194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] PATH_INFO: `/v3/users/f5820d3d7c5c46938dcf72bb05938eeb` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:09:59.990302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5820d3d7c5c46938dcf72bb05938eeb)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:09:59.992254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:00.039105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5820d3d7c5c46938dcf72bb05938eeb'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.039751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5820d3d7c5c46938dcf72bb05938eeb'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.051883 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5820d3d7c5c46938dcf72bb05938eeb'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.058602 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5820d3d7c5c46938dcf72bb05938eeb'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.066121 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5820d3d7c5c46938dcf72bb05938eeb'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.072488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1005/2014] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:09:59 2022] DELETE /identity/v3/users/f5820d3d7c5c46938dcf72bb05938eeb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:10:00.076925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52feb69-4ef0-4a15-b165-9ee94b762d60 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.077430 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52feb69-4ef0-4a15-b165-9ee94b762d60 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.077587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52feb69-4ef0-4a15-b165-9ee94b762d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.097618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f52feb69-4ef0-4a15-b165-9ee94b762d60 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:00.098856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f52feb69-4ef0-4a15-b165-9ee94b762d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:00.157042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1010/2015] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:10:00.166698 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:00.168192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:00.225209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcc712-a2c2-4b94-a88c-0953ad382312 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:00.228843 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61941de5-6cba-4b02-8435-081d5f003656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:00.229442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61941de5-6cba-4b02-8435-081d5f003656 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.229722 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61941de5-6cba-4b02-8435-081d5f003656 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.229991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61941de5-6cba-4b02-8435-081d5f003656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.239536 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61941de5-6cba-4b02-8435-081d5f003656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:00.240958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61941de5-6cba-4b02-8435-081d5f003656 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:00.250708 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1006/2016] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:10:00.514645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52feb69-4ef0-4a15-b165-9ee94b762d60 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:00.516770 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f52feb69-4ef0-4a15-b165-9ee94b762d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:00.529252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52feb69-4ef0-4a15-b165-9ee94b762d60 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:00.534604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:00.535283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.535610 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.535922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] PATH_INFO: `/v3/projects/4dada98c0d7a47e3af3f20cb633ddebc` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.537083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dada98c0d7a47e3af3f20cb633ddebc)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:00.538992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:00.584561 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dada98c0d7a47e3af3f20cb633ddebc', 'request_id': 'req-33935426-f8bf-400e-9ab5-7007bc8b2f23'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.591617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dada98c0d7a47e3af3f20cb633ddebc', 'request_id': 'req-33935426-f8bf-400e-9ab5-7007bc8b2f23'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.598438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dada98c0d7a47e3af3f20cb633ddebc', 'request_id': 'req-33935426-f8bf-400e-9ab5-7007bc8b2f23'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.608619 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] The token cache is being invalidate because project 4dada98c0d7a47e3af3f20cb633ddebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:10:00.608902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.609519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-33935426-f8bf-400e-9ab5-7007bc8b2f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:00.611034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1011/2017] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:10:00 2022] DELETE /identity/v3/projects/4dada98c0d7a47e3af3f20cb633ddebc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:10:00.615930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c54464-03d6-4519-a314-499ce3f1dced None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.616211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c54464-03d6-4519-a314-499ce3f1dced None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.616497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c54464-03d6-4519-a314-499ce3f1dced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.636036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-36c54464-03d6-4519-a314-499ce3f1dced None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:00.637488 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-36c54464-03d6-4519-a314-499ce3f1dced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:00.654806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:00.698142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1007/2018] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:00.707187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5718d-f1a2-4cfd-803c-1a6faf616073 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.707472 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5718d-f1a2-4cfd-803c-1a6faf616073 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.707743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5718d-f1a2-4cfd-803c-1a6faf616073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.727859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-13c5718d-f1a2-4cfd-803c-1a6faf616073 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:00.729054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-13c5718d-f1a2-4cfd-803c-1a6faf616073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:00.736896 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1012/2019] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:00.741030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c54464-03d6-4519-a314-499ce3f1dced None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:00.750668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c54464-03d6-4519-a314-499ce3f1dced None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:00.755467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910a6d2-8fba-4e59-932e-888dea13da19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:00.756076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910a6d2-8fba-4e59-932e-888dea13da19 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.756355 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910a6d2-8fba-4e59-932e-888dea13da19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.756628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910a6d2-8fba-4e59-932e-888dea13da19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.757306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910a6d2-8fba-4e59-932e-888dea13da19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:10:00.757732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910a6d2-8fba-4e59-932e-888dea13da19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:00.759416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910a6d2-8fba-4e59-932e-888dea13da19 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:00.772218 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1008/2020] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:10:00.776453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5718d-f1a2-4cfd-803c-1a6faf616073 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:00.786443 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5718d-f1a2-4cfd-803c-1a6faf616073 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:00.791100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8331aa-b445-4459-b8ad-f71ae1dab28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:00.791716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8331aa-b445-4459-b8ad-f71ae1dab28a admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.791989 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8331aa-b445-4459-b8ad-f71ae1dab28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.792253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8331aa-b445-4459-b8ad-f71ae1dab28a admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.792879 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8331aa-b445-4459-b8ad-f71ae1dab28a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:00.794759 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8331aa-b445-4459-b8ad-f71ae1dab28a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:00.821956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1013/2021] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:10:00 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:10:00.825982 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910a6d2-8fba-4e59-932e-888dea13da19 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:00.836655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910a6d2-8fba-4e59-932e-888dea13da19 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:00.891370 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d8d7d-e0d6-411a-acf7-71c698667a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:00.891951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8d7d-e0d6-411a-acf7-71c698667a36 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.892224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8d7d-e0d6-411a-acf7-71c698667a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.892491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8d7d-e0d6-411a-acf7-71c698667a36 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.893110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d8d7d-e0d6-411a-acf7-71c698667a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:00.895019 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d8d7d-e0d6-411a-acf7-71c698667a36 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:00.916142 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1009/2022] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:10:00 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:10:00.920388 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8331aa-b445-4459-b8ad-f71ae1dab28a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:00.931611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8331aa-b445-4459-b8ad-f71ae1dab28a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:00.936293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f286f-6f46-4e61-a057-b6899d4444bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:00.936910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f286f-6f46-4e61-a057-b6899d4444bd admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.937187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f286f-6f46-4e61-a057-b6899d4444bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.937455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f286f-6f46-4e61-a057-b6899d4444bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.938091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f286f-6f46-4e61-a057-b6899d4444bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:00.939728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f286f-6f46-4e61-a057-b6899d4444bd admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:00.949745 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1014/2023] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:10:00.953596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d8d7d-e0d6-411a-acf7-71c698667a36 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:00.963519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d8d7d-e0d6-411a-acf7-71c698667a36 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:00.968542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6561b81-e017-492c-a894-05ad10cb6e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:00.969123 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6561b81-e017-492c-a894-05ad10cb6e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:00.969399 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6561b81-e017-492c-a894-05ad10cb6e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:00.969696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6561b81-e017-492c-a894-05ad10cb6e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:00.970325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6561b81-e017-492c-a894-05ad10cb6e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:00.971946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6561b81-e017-492c-a894-05ad10cb6e63 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:00.982292 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1010/2024] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:10:00.986756 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f286f-6f46-4e61-a057-b6899d4444bd admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:00.996348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f286f-6f46-4e61-a057-b6899d4444bd admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:01.000909 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b70023-bf5e-4db2-9ebe-983ef742877f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:01.001542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b70023-bf5e-4db2-9ebe-983ef742877f admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:01.001816 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b70023-bf5e-4db2-9ebe-983ef742877f admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:01.002084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b70023-bf5e-4db2-9ebe-983ef742877f admin admin] PATH_INFO: `/v3/projects/bbac7c759b264dba88f7cb7b899cd360/users/91d69e5a88004926904fd83cd63cc61b/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:01.016924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b70023-bf5e-4db2-9ebe-983ef742877f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbac7c759b264dba88f7cb7b899cd360, user_id=91d69e5a88004926904fd83cd63cc61b, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:01.019594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b70023-bf5e-4db2-9ebe-983ef742877f admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:01.033076 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1015/2025] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:10:00 2022] PUT /identity/v3/projects/bbac7c759b264dba88f7cb7b899cd360/users/91d69e5a88004926904fd83cd63cc61b/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:10:01.043258 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c890da-4cd8-4a6a-81b2-825c99ff40ae None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:01.043545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c890da-4cd8-4a6a-81b2-825c99ff40ae None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:01.043821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c890da-4cd8-4a6a-81b2-825c99ff40ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:01.073691 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a6c890da-4cd8-4a6a-81b2-825c99ff40ae None None] MFA Rules not processed for user `91d69e5a88004926904fd83cd63cc61b`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:01.074894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a6c890da-4cd8-4a6a-81b2-825c99ff40ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:01.138021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1011/2026] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:01.149338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b70023-bf5e-4db2-9ebe-983ef742877f admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:01.151176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-28b70023-bf5e-4db2-9ebe-983ef742877f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:01.172877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:01.209025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b70023-bf5e-4db2-9ebe-983ef742877f admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:01.212566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fede960-1ec7-481c-b84e-b3e638ecb73f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:01.213166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fede960-1ec7-481c-b84e-b3e638ecb73f service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:01.213442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fede960-1ec7-481c-b84e-b3e638ecb73f service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:01.213711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fede960-1ec7-481c-b84e-b3e638ecb73f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:01.221733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fede960-1ec7-481c-b84e-b3e638ecb73f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:01.223172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fede960-1ec7-481c-b84e-b3e638ecb73f service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:01.231779 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1016/2027] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:10:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:10:02.589323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0882079-ae81-4a48-acab-0112822ee9af None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:02.589841 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0882079-ae81-4a48-acab-0112822ee9af None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:02.589841 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0882079-ae81-4a48-acab-0112822ee9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:02.608748 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-c0882079-ae81-4a48-acab-0112822ee9af None None] MFA Rules not processed for user `91d69e5a88004926904fd83cd63cc61b`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:02.609866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c0882079-ae81-4a48-acab-0112822ee9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:02.616620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1012/2028] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:11.227944 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=17 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:12.603277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:12.684837 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fede960-1ec7-481c-b84e-b3e638ecb73f service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:12.686259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7fede960-1ec7-481c-b84e-b3e638ecb73f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:12.819170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fede960-1ec7-481c-b84e-b3e638ecb73f service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:12.824152 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5914e53-ae72-4312-886f-32da86b5062a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da958657ac3041848d24a76b76705489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:12.824761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5914e53-ae72-4312-886f-32da86b5062a service placement] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:12.825042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5914e53-ae72-4312-886f-32da86b5062a service placement] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:12.825317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5914e53-ae72-4312-886f-32da86b5062a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:12.827079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f5914e53-ae72-4312-886f-32da86b5062a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:12.839031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5914e53-ae72-4312-886f-32da86b5062a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:12.840757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5914e53-ae72-4312-886f-32da86b5062a service placement] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:12.850504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1017/2029] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:10:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:10:22.846465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=32 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:25.270860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e6cf6-24bb-47be-9e33-d44e75b24f56 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:25.271441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e6cf6-24bb-47be-9e33-d44e75b24f56 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:25.271543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e6cf6-24bb-47be-9e33-d44e75b24f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:25.291943 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-774e6cf6-24bb-47be-9e33-d44e75b24f56 None None] MFA Rules not processed for user `e868bb74b33f467380562aec01a3d3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:25.293192 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-774e6cf6-24bb-47be-9e33-d44e75b24f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:25.348906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1013/2030] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:32.288456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c858202-84fe-449e-84cd-12e0c0e3d004 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:32.289273 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c858202-84fe-449e-84cd-12e0c0e3d004 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:32.289554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c858202-84fe-449e-84cd-12e0c0e3d004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:32.309511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2c858202-84fe-449e-84cd-12e0c0e3d004 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:32.310722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2c858202-84fe-449e-84cd-12e0c0e3d004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:32.369875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1018/2031] 10.222.0.58 () {58 vars in 967 bytes} [Fri Oct 28 19:10:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:10:32.431681 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e6cf6-24bb-47be-9e33-d44e75b24f56 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:32.433300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-774e6cf6-24bb-47be-9e33-d44e75b24f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:32.523162 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e6cf6-24bb-47be-9e33-d44e75b24f56 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:32.528204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d479e03-7bee-4b7f-9753-b27deec02515 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:32.528831 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d479e03-7bee-4b7f-9753-b27deec02515 service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:32.529112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d479e03-7bee-4b7f-9753-b27deec02515 service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:32.529388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d479e03-7bee-4b7f-9753-b27deec02515 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:32.539308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d479e03-7bee-4b7f-9753-b27deec02515 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:32.541245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d479e03-7bee-4b7f-9753-b27deec02515 service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:32.559881 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1014/2032] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:10:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:10:33.329689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9af93e-a696-4681-bc9d-ee513ff8e03e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:33.330172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9af93e-a696-4681-bc9d-ee513ff8e03e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:33.330374 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9af93e-a696-4681-bc9d-ee513ff8e03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:33.349636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-0d9af93e-a696-4681-bc9d-ee513ff8e03e None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:33.350919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0d9af93e-a696-4681-bc9d-ee513ff8e03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:33.358355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1019/2033] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:10:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:33.370677 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d479e03-7bee-4b7f-9753-b27deec02515 service swift] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:33.381613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d479e03-7bee-4b7f-9753-b27deec02515 service swift] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:33.388822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2502685-b308-45ae-bdf1-b0cbfd7bd674 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:33.389414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2502685-b308-45ae-bdf1-b0cbfd7bd674 service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:33.389688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2502685-b308-45ae-bdf1-b0cbfd7bd674 service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:33.390100 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2502685-b308-45ae-bdf1-b0cbfd7bd674 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:33.399485 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2502685-b308-45ae-bdf1-b0cbfd7bd674 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:33.401084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2502685-b308-45ae-bdf1-b0cbfd7bd674 service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:33.409868 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1015/2034] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:10:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:10:33.965066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eeda4d-6a59-49af-ace2-57b5bac21235 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:33.965355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eeda4d-6a59-49af-ace2-57b5bac21235 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:33.965628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eeda4d-6a59-49af-ace2-57b5bac21235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:33.986458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-16eeda4d-6a59-49af-ace2-57b5bac21235 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:33.990504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-16eeda4d-6a59-49af-ace2-57b5bac21235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:33.998706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1020/2035] 10.222.0.58 () {58 vars in 967 bytes} [Fri Oct 28 19:10:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:34.013087 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2502685-b308-45ae-bdf1-b0cbfd7bd674 service swift] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:34.024465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2502685-b308-45ae-bdf1-b0cbfd7bd674 service swift] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:34.030232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ee22b-e3d4-4ae2-8e9f-f544553dfe4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:34.031424 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ee22b-e3d4-4ae2-8e9f-f544553dfe4e service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:34.031662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ee22b-e3d4-4ae2-8e9f-f544553dfe4e service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:34.031933 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ee22b-e3d4-4ae2-8e9f-f544553dfe4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:34.042081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ee22b-e3d4-4ae2-8e9f-f544553dfe4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:34.044068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ee22b-e3d4-4ae2-8e9f-f544553dfe4e service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:34.055981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1016/2036] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:10:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:10:34.343506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1acd-baea-49ce-b641-bdab4145b1b8 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:34.344348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1acd-baea-49ce-b641-bdab4145b1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:34.344856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1acd-baea-49ce-b641-bdab4145b1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:34.372328 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-0e7f1acd-baea-49ce-b641-bdab4145b1b8 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:34.374107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0e7f1acd-baea-49ce-b641-bdab4145b1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:34.383265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1021/2037] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:10:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:10:34.398381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ee22b-e3d4-4ae2-8e9f-f544553dfe4e service swift] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:34.410783 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ee22b-e3d4-4ae2-8e9f-f544553dfe4e service swift] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:34.417026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf040d98-4d5e-4d3a-86b9-c3501111378c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:34.417870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf040d98-4d5e-4d3a-86b9-c3501111378c service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:34.418418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf040d98-4d5e-4d3a-86b9-c3501111378c service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:34.418907 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf040d98-4d5e-4d3a-86b9-c3501111378c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:34.430245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf040d98-4d5e-4d3a-86b9-c3501111378c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:34.432196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf040d98-4d5e-4d3a-86b9-c3501111378c service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:34.443477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1017/2038] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:10:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:10:34.808145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4f65b-455c-4d35-b851-ddd2f79c2b7d None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:34.808488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4f65b-455c-4d35-b851-ddd2f79c2b7d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:34.808982 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4f65b-455c-4d35-b851-ddd2f79c2b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:34.834034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f7e4f65b-455c-4d35-b851-ddd2f79c2b7d None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:34.835650 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f7e4f65b-455c-4d35-b851-ddd2f79c2b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:34.856236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1022/2039] 10.222.0.58 () {58 vars in 967 bytes} [Fri Oct 28 19:10:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:34.864970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf040d98-4d5e-4d3a-86b9-c3501111378c service swift] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:34.878587 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf040d98-4d5e-4d3a-86b9-c3501111378c service swift] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:34.884340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16bbe8-902b-4a7a-b1ef-87758fef016a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:34.885495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16bbe8-902b-4a7a-b1ef-87758fef016a service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:34.885921 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16bbe8-902b-4a7a-b1ef-87758fef016a service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:34.886531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16bbe8-902b-4a7a-b1ef-87758fef016a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:34.902492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16bbe8-902b-4a7a-b1ef-87758fef016a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:34.904380 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16bbe8-902b-4a7a-b1ef-87758fef016a service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:34.929234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1018/2040] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:10:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:10:37.279068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4f65b-455c-4d35-b851-ddd2f79c2b7d None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:37.417037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4f65b-455c-4d35-b851-ddd2f79c2b7d None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:37.422522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:37.423550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:37.424089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:37.424854 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] PATH_INFO: `/v3/users/91d69e5a88004926904fd83cd63cc61b` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:37.426797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d69e5a88004926904fd83cd63cc61b)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:37.429203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:37.506618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d69e5a88004926904fd83cd63cc61b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:37.507410 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d69e5a88004926904fd83cd63cc61b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:37.523425 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d69e5a88004926904fd83cd63cc61b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:37.533417 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d69e5a88004926904fd83cd63cc61b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:37.535248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d69e5a88004926904fd83cd63cc61b'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:37.549902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:37.559449 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1023/2041] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:10:37 2022] DELETE /identity/v3/users/91d69e5a88004926904fd83cd63cc61b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:10:37.561136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b913e-9bf8-4334-becc-dc7f5fccebfb None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:37.561624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b913e-9bf8-4334-becc-dc7f5fccebfb None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:37.561721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b913e-9bf8-4334-becc-dc7f5fccebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:37.582760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-886b913e-9bf8-4334-becc-dc7f5fccebfb None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:37.584029 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-886b913e-9bf8-4334-becc-dc7f5fccebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:37.623712 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=100 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:37.647092 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1019/2042] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:37.659588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:37.662287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:37.757881 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279dc86-4851-4ddf-811c-c74c6676f96c admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:37.762088 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e8424-0ffb-41db-a1c3-3cfa905b78f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:37.762715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e8424-0ffb-41db-a1c3-3cfa905b78f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:37.763018 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e8424-0ffb-41db-a1c3-3cfa905b78f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:37.763287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e8424-0ffb-41db-a1c3-3cfa905b78f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:37.779504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e8424-0ffb-41db-a1c3-3cfa905b78f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:37.781011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e8424-0ffb-41db-a1c3-3cfa905b78f8 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:37.791251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1024/2043] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:10:38.085953 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b913e-9bf8-4334-becc-dc7f5fccebfb None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:38.087412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-886b913e-9bf8-4334-becc-dc7f5fccebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:38.101728 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b913e-9bf8-4334-becc-dc7f5fccebfb None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:38.106346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:38.107010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.107503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.107831 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] PATH_INFO: `/v3/projects/bbac7c759b264dba88f7cb7b899cd360` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.110197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbac7c759b264dba88f7cb7b899cd360)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:38.113810 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:38.165034 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbac7c759b264dba88f7cb7b899cd360', 'request_id': 'req-909e4574-c760-48cc-b8ff-87bd077a6a2c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:38.171471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbac7c759b264dba88f7cb7b899cd360', 'request_id': 'req-909e4574-c760-48cc-b8ff-87bd077a6a2c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:38.178766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbac7c759b264dba88f7cb7b899cd360', 'request_id': 'req-909e4574-c760-48cc-b8ff-87bd077a6a2c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:38.190639 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] The token cache is being invalidate because project bbac7c759b264dba88f7cb7b899cd360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:10:38.190925 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:38.191591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-909e4574-c760-48cc-b8ff-87bd077a6a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:38.193084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1020/2044] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:10:38 2022] DELETE /identity/v3/projects/bbac7c759b264dba88f7cb7b899cd360 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:10:38.198014 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff05dcb-1611-4f3e-9f8c-f0c342dfb102 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.198421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff05dcb-1611-4f3e-9f8c-f0c342dfb102 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.198760 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff05dcb-1611-4f3e-9f8c-f0c342dfb102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.219392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-9ff05dcb-1611-4f3e-9f8c-f0c342dfb102 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:38.220595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9ff05dcb-1611-4f3e-9f8c-f0c342dfb102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:38.280881 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1025/2045] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:38.288427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2e636-4a88-4e4b-8d8b-36917307fccf None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.288714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2e636-4a88-4e4b-8d8b-36917307fccf None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.288989 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2e636-4a88-4e4b-8d8b-36917307fccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.308916 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-c2a2e636-4a88-4e4b-8d8b-36917307fccf None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:38.310155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-c2a2e636-4a88-4e4b-8d8b-36917307fccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:38.318865 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1021/2046] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:38.323071 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff05dcb-1611-4f3e-9f8c-f0c342dfb102 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:38.333872 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff05dcb-1611-4f3e-9f8c-f0c342dfb102 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:38.338399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3c3fc-9e29-4a49-8a28-4ae5a050fa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:38.339021 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3c3fc-9e29-4a49-8a28-4ae5a050fa72 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.339303 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3c3fc-9e29-4a49-8a28-4ae5a050fa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.339573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3c3fc-9e29-4a49-8a28-4ae5a050fa72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.340247 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3c3fc-9e29-4a49-8a28-4ae5a050fa72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:10:38.340546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3c3fc-9e29-4a49-8a28-4ae5a050fa72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:38.342134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3c3fc-9e29-4a49-8a28-4ae5a050fa72 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:38.357348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1026/2047] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:10:38.358396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2e636-4a88-4e4b-8d8b-36917307fccf None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:38.367556 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2e636-4a88-4e4b-8d8b-36917307fccf None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:38.372305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53577475-5e9e-4df6-8c3e-2b3dfb2c802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:38.372905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53577475-5e9e-4df6-8c3e-2b3dfb2c802e admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.373184 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53577475-5e9e-4df6-8c3e-2b3dfb2c802e admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.373456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53577475-5e9e-4df6-8c3e-2b3dfb2c802e admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.374094 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53577475-5e9e-4df6-8c3e-2b3dfb2c802e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:38.376593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53577475-5e9e-4df6-8c3e-2b3dfb2c802e admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:38.406382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1022/2048] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:10:38 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:10:38.410659 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3c3fc-9e29-4a49-8a28-4ae5a050fa72 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:38.425403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3c3fc-9e29-4a49-8a28-4ae5a050fa72 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:38.489325 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61a41-2e26-40b1-99b2-4c1e8fddbe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:38.490077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61a41-2e26-40b1-99b2-4c1e8fddbe17 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.490361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61a41-2e26-40b1-99b2-4c1e8fddbe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.490629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61a41-2e26-40b1-99b2-4c1e8fddbe17 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.491401 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db61a41-2e26-40b1-99b2-4c1e8fddbe17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:38.493503 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db61a41-2e26-40b1-99b2-4c1e8fddbe17 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:38.523053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1027/2049] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:10:38 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:10:38.527203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53577475-5e9e-4df6-8c3e-2b3dfb2c802e admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:38.538642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53577475-5e9e-4df6-8c3e-2b3dfb2c802e admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:38.543628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161188b-5d83-4cea-be4e-c59d73620279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:38.544247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161188b-5d83-4cea-be4e-c59d73620279 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.544525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161188b-5d83-4cea-be4e-c59d73620279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.544795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161188b-5d83-4cea-be4e-c59d73620279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.545593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161188b-5d83-4cea-be4e-c59d73620279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:38.547934 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161188b-5d83-4cea-be4e-c59d73620279 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:38.558690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1023/2050] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:10:38.562710 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61a41-2e26-40b1-99b2-4c1e8fddbe17 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:38.573516 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61a41-2e26-40b1-99b2-4c1e8fddbe17 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:38.578632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0125c71-4e0d-4421-8700-d3de277eded2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:38.579350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0125c71-4e0d-4421-8700-d3de277eded2 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.579622 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0125c71-4e0d-4421-8700-d3de277eded2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.580022 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0125c71-4e0d-4421-8700-d3de277eded2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.580734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0125c71-4e0d-4421-8700-d3de277eded2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:38.582404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0125c71-4e0d-4421-8700-d3de277eded2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:38.594961 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1028/2051] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:10:38.599347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161188b-5d83-4cea-be4e-c59d73620279 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:38.610586 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161188b-5d83-4cea-be4e-c59d73620279 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:38.615331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281c19-0dce-4e63-a8ea-88fde48111eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:38.616038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281c19-0dce-4e63-a8ea-88fde48111eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.616418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281c19-0dce-4e63-a8ea-88fde48111eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.616733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281c19-0dce-4e63-a8ea-88fde48111eb admin admin] PATH_INFO: `/v3/projects/10e5356491f54ddeb10f71d5c927e753/users/9a4a47b7a9284cc8b44c8c732e5b6647/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.633961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f281c19-0dce-4e63-a8ea-88fde48111eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e5356491f54ddeb10f71d5c927e753, user_id=9a4a47b7a9284cc8b44c8c732e5b6647, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:38.637997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f281c19-0dce-4e63-a8ea-88fde48111eb admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:38.660991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1024/2052] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:10:38 2022] PUT /identity/v3/projects/10e5356491f54ddeb10f71d5c927e753/users/9a4a47b7a9284cc8b44c8c732e5b6647/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:10:38.662715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9b323-4dc0-4c4c-b4d6-6e0df3929bcd None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.663003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9b323-4dc0-4c4c-b4d6-6e0df3929bcd None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.663327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9b323-4dc0-4c4c-b4d6-6e0df3929bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.677469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:38.685684 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f6a9b323-4dc0-4c4c-b4d6-6e0df3929bcd None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:38.687910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f6a9b323-4dc0-4c4c-b4d6-6e0df3929bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:38.762478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1029/2053] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:38.774114 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281c19-0dce-4e63-a8ea-88fde48111eb admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:38.775946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6f281c19-0dce-4e63-a8ea-88fde48111eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:38.844492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281c19-0dce-4e63-a8ea-88fde48111eb admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:38.848714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45396f-39e4-488c-abb8-4d0f21496c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:38.849534 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45396f-39e4-488c-abb8-4d0f21496c80 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:38.850017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45396f-39e4-488c-abb8-4d0f21496c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:38.850490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45396f-39e4-488c-abb8-4d0f21496c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:38.862782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45396f-39e4-488c-abb8-4d0f21496c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:38.864502 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45396f-39e4-488c-abb8-4d0f21496c80 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:38.875204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1025/2054] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:10:42.383077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9b323-4dc0-4c4c-b4d6-6e0df3929bcd None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:42.385312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f6a9b323-4dc0-4c4c-b4d6-6e0df3929bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:42.407703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9b323-4dc0-4c4c-b4d6-6e0df3929bcd None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:42.419033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:42.419844 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:42.420035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:42.420517 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] PATH_INFO: `/v3/users/fec54d8c1ea24cc8bbbeb6fef1fd2028` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:42.428475 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec54d8c1ea24cc8bbbeb6fef1fd2028)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:42.432762 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:42.519456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec54d8c1ea24cc8bbbeb6fef1fd2028'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:42.520859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec54d8c1ea24cc8bbbeb6fef1fd2028'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:42.539795 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec54d8c1ea24cc8bbbeb6fef1fd2028'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:42.551147 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec54d8c1ea24cc8bbbeb6fef1fd2028'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:42.553617 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec54d8c1ea24cc8bbbeb6fef1fd2028'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:42.575807 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1030/2055] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:10:42 2022] DELETE /identity/v3/users/fec54d8c1ea24cc8bbbeb6fef1fd2028 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:10:42.973346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45396f-39e4-488c-abb8-4d0f21496c80 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:42.982524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:42.989557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45396f-39e4-488c-abb8-4d0f21496c80 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:43.132641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:43.135104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:43.135628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:43.139234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] PATH_INFO: `/v3/projects/aa4bbb7ba7f14bd9bcaaa0ea461322a6` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:43.141970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4bbb7ba7f14bd9bcaaa0ea461322a6)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:43.145678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:43.217329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4bbb7ba7f14bd9bcaaa0ea461322a6', 'request_id': 'req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.227964 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4bbb7ba7f14bd9bcaaa0ea461322a6', 'request_id': 'req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.253193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4bbb7ba7f14bd9bcaaa0ea461322a6', 'request_id': 'req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.276430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] The token cache is being invalidate because project aa4bbb7ba7f14bd9bcaaa0ea461322a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:10:43.278458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.280043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.284842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1026/2056] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:10:42 2022] DELETE /identity/v3/projects/aa4bbb7ba7f14bd9bcaaa0ea461322a6 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:10:43.290442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:43.292731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:43.311353 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55408d8a-88ff-487b-b347-6d800d570346 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:43.474437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:43.475133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:43.475418 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:43.475687 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] PATH_INFO: `/v3/users/7f7c84df58964308a4a79228338b2703` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:43.477898 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7c84df58964308a4a79228338b2703)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:43.481678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:43.536194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:43.579097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7c84df58964308a4a79228338b2703'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.580743 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7c84df58964308a4a79228338b2703'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.609943 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7c84df58964308a4a79228338b2703'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.622131 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7c84df58964308a4a79228338b2703'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.623170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-0f95f8be-0717-4dff-b800-04cdaad837bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7c84df58964308a4a79228338b2703'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:43.671097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1031/2057] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:10:43 2022] DELETE /identity/v3/users/7f7c84df58964308a4a79228338b2703 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:10:43.953785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:43.955323 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:43.972946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ba1b1-f11e-4ac6-85f5-9619bd28ed95 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:44.047335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:44.048188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:44.048858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:44.049199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] PATH_INFO: `/v3/projects/8ac2b0054b6b40a7ae811f72227670d3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:44.050760 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac2b0054b6b40a7ae811f72227670d3)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:44.052821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:44.117855 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:44.130789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac2b0054b6b40a7ae811f72227670d3', 'request_id': 'req-a37ece99-fead-413f-87f9-7a5e17018901'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:44.138899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac2b0054b6b40a7ae811f72227670d3', 'request_id': 'req-a37ece99-fead-413f-87f9-7a5e17018901'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:44.149516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac2b0054b6b40a7ae811f72227670d3', 'request_id': 'req-a37ece99-fead-413f-87f9-7a5e17018901'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:44.167819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] The token cache is being invalidate because project 8ac2b0054b6b40a7ae811f72227670d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:10:44.168131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:44.168766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:10:44.185605 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1027/2058] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:10:43 2022] DELETE /identity/v3/projects/8ac2b0054b6b40a7ae811f72227670d3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:10:44.789585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fec701-c62f-440f-a78b-7779277d149f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:44.790057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fec701-c62f-440f-a78b-7779277d149f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:44.790138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fec701-c62f-440f-a78b-7779277d149f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:44.828569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-80fec701-c62f-440f-a78b-7779277d149f None None] MFA Rules not processed for user `9a4a47b7a9284cc8b44c8c732e5b6647`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:44.830697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-80fec701-c62f-440f-a78b-7779277d149f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:44.906505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1032/2059] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:44.910773 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:44.912347 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:44.984188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ece99-fead-413f-87f9-7a5e17018901 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:44.988930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd4e39-4458-4996-ad98-c4dfa3a3533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:44.989620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4e39-4458-4996-ad98-c4dfa3a3533c admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:44.989898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4e39-4458-4996-ad98-c4dfa3a3533c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:44.990163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4e39-4458-4996-ad98-c4dfa3a3533c admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:44.990791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd4e39-4458-4996-ad98-c4dfa3a3533c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:44.992724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd4e39-4458-4996-ad98-c4dfa3a3533c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:45.023029 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1028/2060] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:10:44 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:10:45.026818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fec701-c62f-440f-a78b-7779277d149f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:45.038489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fec701-c62f-440f-a78b-7779277d149f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:45.099934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3d50-7ce1-413a-bd9d-2e2d58718b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:45.100554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3d50-7ce1-413a-bd9d-2e2d58718b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:45.100831 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3d50-7ce1-413a-bd9d-2e2d58718b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:45.101101 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3d50-7ce1-413a-bd9d-2e2d58718b9a admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:45.101744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3d50-7ce1-413a-bd9d-2e2d58718b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:45.103883 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3d50-7ce1-413a-bd9d-2e2d58718b9a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:45.132233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1033/2061] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:10:45 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:10:45.135464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd4e39-4458-4996-ad98-c4dfa3a3533c admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:45.158062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd4e39-4458-4996-ad98-c4dfa3a3533c admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:45.162880 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd1973-e044-4e83-aacb-a1c7d5d6bde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:45.163630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd1973-e044-4e83-aacb-a1c7d5d6bde8 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:45.163960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd1973-e044-4e83-aacb-a1c7d5d6bde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:45.164171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd1973-e044-4e83-aacb-a1c7d5d6bde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:45.164838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd1973-e044-4e83-aacb-a1c7d5d6bde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:45.166456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd1973-e044-4e83-aacb-a1c7d5d6bde8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:45.176742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1029/2062] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:10:45.182669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3d50-7ce1-413a-bd9d-2e2d58718b9a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:45.195594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3d50-7ce1-413a-bd9d-2e2d58718b9a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:45.201089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab032ff1-7b96-457d-8119-2dd5aa262a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:45.201624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab032ff1-7b96-457d-8119-2dd5aa262a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:45.201904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab032ff1-7b96-457d-8119-2dd5aa262a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:45.202736 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab032ff1-7b96-457d-8119-2dd5aa262a84 admin admin] PATH_INFO: `/v3/projects/b2f0b7258a9946a79ebe4137f2347b94/users/2ccab25312874a6ba977fc50be0a5b6b/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:45.219100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab032ff1-7b96-457d-8119-2dd5aa262a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f0b7258a9946a79ebe4137f2347b94, user_id=2ccab25312874a6ba977fc50be0a5b6b, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:45.222062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab032ff1-7b96-457d-8119-2dd5aa262a84 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:45.224437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:45.237822 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1034/2063] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:10:45 2022] PUT /identity/v3/projects/b2f0b7258a9946a79ebe4137f2347b94/users/2ccab25312874a6ba977fc50be0a5b6b/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:10:45.241597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd1973-e044-4e83-aacb-a1c7d5d6bde8 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:45.252725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd1973-e044-4e83-aacb-a1c7d5d6bde8 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:45.314182 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e5ffe-b1b2-4480-82f4-b70b894ab19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:45.314937 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e5ffe-b1b2-4480-82f4-b70b894ab19d admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:45.315250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e5ffe-b1b2-4480-82f4-b70b894ab19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:45.315519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e5ffe-b1b2-4480-82f4-b70b894ab19d admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:45.316360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e5ffe-b1b2-4480-82f4-b70b894ab19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:45.317967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e5ffe-b1b2-4480-82f4-b70b894ab19d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:45.329314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1030/2064] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:10:45.333477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab032ff1-7b96-457d-8119-2dd5aa262a84 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:45.344591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab032ff1-7b96-457d-8119-2dd5aa262a84 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:45.349409 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafc8be-a2e5-42c1-a3d1-6852a4e4cc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:45.350083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafc8be-a2e5-42c1-a3d1-6852a4e4cc2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:45.350361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafc8be-a2e5-42c1-a3d1-6852a4e4cc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:45.350634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafc8be-a2e5-42c1-a3d1-6852a4e4cc2b admin admin] PATH_INFO: `/v3/projects/b2f0b7258a9946a79ebe4137f2347b94/users/2ccab25312874a6ba977fc50be0a5b6b/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:45.352714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafc8be-a2e5-42c1-a3d1-6852a4e4cc2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f0b7258a9946a79ebe4137f2347b94, user_id=2ccab25312874a6ba977fc50be0a5b6b, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:45.355483 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafc8be-a2e5-42c1-a3d1-6852a4e4cc2b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:45.369328 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1035/2065] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:10:45 2022] PUT /identity/v3/projects/b2f0b7258a9946a79ebe4137f2347b94/users/2ccab25312874a6ba977fc50be0a5b6b/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:10:49.727636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac28b1-f080-4013-8543-19860c0d4544 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:49.736319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac28b1-f080-4013-8543-19860c0d4544 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:49.736868 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac28b1-f080-4013-8543-19860c0d4544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:49.785432 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-59ac28b1-f080-4013-8543-19860c0d4544 None None] MFA Rules not processed for user `2ccab25312874a6ba977fc50be0a5b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:49.787339 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-59ac28b1-f080-4013-8543-19860c0d4544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:49.819638 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:10:49.868273 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1031/2066] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:49.881967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafc8be-a2e5-42c1-a3d1-6852a4e4cc2b admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:49.883739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ecafc8be-a2e5-42c1-a3d1-6852a4e4cc2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:49.960127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafc8be-a2e5-42c1-a3d1-6852a4e4cc2b admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:49.965576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee1521-fb68-449d-a649-e57e6d950819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:49.966178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee1521-fb68-449d-a649-e57e6d950819 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:49.966457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee1521-fb68-449d-a649-e57e6d950819 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:49.966721 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee1521-fb68-449d-a649-e57e6d950819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:50.024271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee1521-fb68-449d-a649-e57e6d950819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:50.025856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee1521-fb68-449d-a649-e57e6d950819 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:50.034731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1036/2067] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:10:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:10:51.125226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bf632-6b0b-45c5-b6e8-5e429e5ad981 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:51.125727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bf632-6b0b-45c5-b6e8-5e429e5ad981 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:51.125877 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bf632-6b0b-45c5-b6e8-5e429e5ad981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:51.145788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-810bf632-6b0b-45c5-b6e8-5e429e5ad981 None None] MFA Rules not processed for user `9a4a47b7a9284cc8b44c8c732e5b6647`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:51.147076 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-810bf632-6b0b-45c5-b6e8-5e429e5ad981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:51.154105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1032/2068] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:51.932391 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab29d0-f1a9-41b1-9f72-593045fc8d33 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:51.932930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab29d0-f1a9-41b1-9f72-593045fc8d33 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:51.932930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab29d0-f1a9-41b1-9f72-593045fc8d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:51.953689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-43ab29d0-f1a9-41b1-9f72-593045fc8d33 None None] MFA Rules not processed for user `c8ca34fc5c5743d0b5e3f4f97ae285e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:10:51.954895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-43ab29d0-f1a9-41b1-9f72-593045fc8d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:52.019813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1037/2069] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:10:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:10:56.033704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bf632-6b0b-45c5-b6e8-5e429e5ad981 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:10:56.035254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-810bf632-6b0b-45c5-b6e8-5e429e5ad981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:56.093020 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bf632-6b0b-45c5-b6e8-5e429e5ad981 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:10:56.097808 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:10:56.098429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:10:56.098731 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:10:56.098968 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:10:56.100744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:10:56.114836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:10:56.116283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:10:56.128426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1033/2070] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:10:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:02.011045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:INSERT=2,keystone:SELECT=88 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:11:06.122226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=47 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:11:07.511051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab29d0-f1a9-41b1-9f72-593045fc8d33 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:07.512515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-43ab29d0-f1a9-41b1-9f72-593045fc8d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:07.567499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab29d0-f1a9-41b1-9f72-593045fc8d33 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:07.571054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f2abe-ad15-4556-bc3e-a2cd5ea02e20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:07.571648 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2abe-ad15-4556-bc3e-a2cd5ea02e20 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:07.571925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2abe-ad15-4556-bc3e-a2cd5ea02e20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:07.572200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2abe-ad15-4556-bc3e-a2cd5ea02e20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:07.573869 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f50f2abe-ad15-4556-bc3e-a2cd5ea02e20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:07.587225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f2abe-ad15-4556-bc3e-a2cd5ea02e20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:07.588640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f2abe-ad15-4556-bc3e-a2cd5ea02e20 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:07.599587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1038/2071] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:14.231952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:14.233418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:14.249057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301000e-b737-405a-ae8f-58c3daac35ca service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:14.254870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3b55b-fdb4-48bf-a5d2-0a5fd1f8f959 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:14.255529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3b55b-fdb4-48bf-a5d2-0a5fd1f8f959 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:14.255807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3b55b-fdb4-48bf-a5d2-0a5fd1f8f959 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:14.256078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3b55b-fdb4-48bf-a5d2-0a5fd1f8f959 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:14.257887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-04f3b55b-fdb4-48bf-a5d2-0a5fd1f8f959 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:14.357305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3b55b-fdb4-48bf-a5d2-0a5fd1f8f959 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:14.359080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3b55b-fdb4-48bf-a5d2-0a5fd1f8f959 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:14.383585 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1034/2072] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:17.594464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=26 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:11:18.561311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bba58-109f-4bad-bab4-4e14243d366a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:18.561599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bba58-109f-4bad-bab4-4e14243d366a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:18.561871 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bba58-109f-4bad-bab4-4e14243d366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:18.582371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-9c8bba58-109f-4bad-bab4-4e14243d366a None None] MFA Rules not processed for user `a25c8936b368477880f429e818a321fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:11:18.583656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9c8bba58-109f-4bad-bab4-4e14243d366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:18.590367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1039/2073] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:11:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:11:18.903992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3b55b-fdb4-48bf-a5d2-0a5fd1f8f959 service nova] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:18.913116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3b55b-fdb4-48bf-a5d2-0a5fd1f8f959 service nova] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:18.916667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15950aa9-3311-43c2-af22-14883cf2bfb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:18.917253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15950aa9-3311-43c2-af22-14883cf2bfb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:18.917529 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15950aa9-3311-43c2-af22-14883cf2bfb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:18.917801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15950aa9-3311-43c2-af22-14883cf2bfb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:18.919504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-15950aa9-3311-43c2-af22-14883cf2bfb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:18.928671 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15950aa9-3311-43c2-af22-14883cf2bfb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:18.930227 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15950aa9-3311-43c2-af22-14883cf2bfb2 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:18.937997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1035/2074] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:19.300232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bba58-109f-4bad-bab4-4e14243d366a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:19.301600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9c8bba58-109f-4bad-bab4-4e14243d366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:19.384263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bba58-109f-4bad-bab4-4e14243d366a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:19.389415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da958657ac3041848d24a76b76705489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:19.390014 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:19.390288 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:19.390556 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:19.392352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:19.404642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:19.406382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:19.419754 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1040/2075] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:19.893138 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15950aa9-3311-43c2-af22-14883cf2bfb2 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:19.905799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15950aa9-3311-43c2-af22-14883cf2bfb2 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:19.911900 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb053a-9c2c-450c-9eb6-579b302f0bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:19.912422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb053a-9c2c-450c-9eb6-579b302f0bb4 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:19.912700 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb053a-9c2c-450c-9eb6-579b302f0bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:19.912967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb053a-9c2c-450c-9eb6-579b302f0bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:19.914638 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-72eb053a-9c2c-450c-9eb6-579b302f0bb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:19.971174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb053a-9c2c-450c-9eb6-579b302f0bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:19.972781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb053a-9c2c-450c-9eb6-579b302f0bb4 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:19.981717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1036/2076] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:11:22.170654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:22.172269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:22.184133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab8707-8d5c-4c95-a121-e3e6a9cfdbe4 service placement] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:22.187785 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef6d73-100b-4405-ab55-06f6647c8ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:22.188471 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6d73-100b-4405-ab55-06f6647c8ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:22.188752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6d73-100b-4405-ab55-06f6647c8ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:22.189027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6d73-100b-4405-ab55-06f6647c8ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:22.190792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-bfef6d73-100b-4405-ab55-06f6647c8ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:22.200954 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef6d73-100b-4405-ab55-06f6647c8ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:22.202395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef6d73-100b-4405-ab55-06f6647c8ba8 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:22.211477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1041/2077] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:26.926046 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb053a-9c2c-450c-9eb6-579b302f0bb4 service nova] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:26.935826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb053a-9c2c-450c-9eb6-579b302f0bb4 service nova] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:26.939410 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc681d9-016c-4f05-adcc-0535551790ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:26.940017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc681d9-016c-4f05-adcc-0535551790ea service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:26.940292 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc681d9-016c-4f05-adcc-0535551790ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:26.940559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc681d9-016c-4f05-adcc-0535551790ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:26.950106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc681d9-016c-4f05-adcc-0535551790ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:26.951610 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc681d9-016c-4f05-adcc-0535551790ea service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:26.962835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1037/2078] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:28.672408 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef6d73-100b-4405-ab55-06f6647c8ba8 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:28.692188 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef6d73-100b-4405-ab55-06f6647c8ba8 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:28.698278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca7525-1f23-4328-8d44-272acde5b148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:28.699272 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca7525-1f23-4328-8d44-272acde5b148 service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:28.699877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca7525-1f23-4328-8d44-272acde5b148 service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:28.700374 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca7525-1f23-4328-8d44-272acde5b148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:28.702348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-43ca7525-1f23-4328-8d44-272acde5b148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:28.715268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca7525-1f23-4328-8d44-272acde5b148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:28.716911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca7525-1f23-4328-8d44-272acde5b148 service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:28.728404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1042/2079] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:33.327363 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc681d9-016c-4f05-adcc-0535551790ea service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:33.339827 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc681d9-016c-4f05-adcc-0535551790ea service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:33.345396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:33.345987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:33.346266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:33.346538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:33.348298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:33.360661 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:33.362781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:33.374426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1038/2080] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:35.949858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc11e11-f961-48fe-ac9a-c2d0edb3f34b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:35.950335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc11e11-f961-48fe-ac9a-c2d0edb3f34b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:35.950431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc11e11-f961-48fe-ac9a-c2d0edb3f34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:35.970918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-dfc11e11-f961-48fe-ac9a-c2d0edb3f34b None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:11:35.972174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-dfc11e11-f961-48fe-ac9a-c2d0edb3f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:36.028003 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1043/2081] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:11:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:11:36.073322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:36.074759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:36.125441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:11:36.153694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a5ffd-6ffb-4ea9-a340-bb99e7f3297a service nova] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:36.158669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32563770-c086-427f-a872-c0813291d381 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:36.159290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32563770-c086-427f-a872-c0813291d381 service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:36.159561 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32563770-c086-427f-a872-c0813291d381 service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:36.159823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32563770-c086-427f-a872-c0813291d381 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:36.167811 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32563770-c086-427f-a872-c0813291d381 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:36.169424 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32563770-c086-427f-a872-c0813291d381 service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:36.177360 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1039/2082] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:11:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:11:36.216400 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc11e11-f961-48fe-ac9a-c2d0edb3f34b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:36.218315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-dfc11e11-f961-48fe-ac9a-c2d0edb3f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:36.230271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc11e11-f961-48fe-ac9a-c2d0edb3f34b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:36.234066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465226f-df9c-4cf0-af6a-fd8e0dfbac1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:36.234793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465226f-df9c-4cf0-af6a-fd8e0dfbac1b service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:36.235214 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465226f-df9c-4cf0-af6a-fd8e0dfbac1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:36.235582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465226f-df9c-4cf0-af6a-fd8e0dfbac1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:36.238317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8465226f-df9c-4cf0-af6a-fd8e0dfbac1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:36.260710 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465226f-df9c-4cf0-af6a-fd8e0dfbac1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:36.262457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465226f-df9c-4cf0-af6a-fd8e0dfbac1b service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:36.273353 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1044/2083] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:11:37.498463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32563770-c086-427f-a872-c0813291d381 service swift] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:37.515917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32563770-c086-427f-a872-c0813291d381 service swift] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:37.523353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b67d9d-b263-4e6c-8e27-21d50e090176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:37.524268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67d9d-b263-4e6c-8e27-21d50e090176 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:37.524889 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67d9d-b263-4e6c-8e27-21d50e090176 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:37.525330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67d9d-b263-4e6c-8e27-21d50e090176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:37.527415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-15b67d9d-b263-4e6c-8e27-21d50e090176 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:37.544415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b67d9d-b263-4e6c-8e27-21d50e090176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:37.546497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b67d9d-b263-4e6c-8e27-21d50e090176 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:37.571887 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1040/2084] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:46.268320 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=78 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:11:47.554716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=15 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:11:50.442140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3b049-ca22-44b2-9989-e5eef3d635a7 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:50.442679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3b049-ca22-44b2-9989-e5eef3d635a7 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:50.442881 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3b049-ca22-44b2-9989-e5eef3d635a7 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:50.444308 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1045/2085] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:11:50.450166 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2471b-9b4b-498b-baae-21b28844e76b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:50.450996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2471b-9b4b-498b-baae-21b28844e76b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:50.451280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2471b-9b4b-498b-baae-21b28844e76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:50.474540 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-37b2471b-9b4b-498b-baae-21b28844e76b None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:11:50.475812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-37b2471b-9b4b-498b-baae-21b28844e76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:50.486435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1041/2086] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:11:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:11:50.499317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3b049-ca22-44b2-9989-e5eef3d635a7 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:50.501598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-75d3b049-ca22-44b2-9989-e5eef3d635a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:50.562858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3b049-ca22-44b2-9989-e5eef3d635a7 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:50.569921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f80af-5ce1-4bef-be49-a6caf4da57c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8ec16bd9254b23bfd2b60551af9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:50.570577 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f80af-5ce1-4bef-be49-a6caf4da57c8 service glance] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:50.570863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f80af-5ce1-4bef-be49-a6caf4da57c8 service glance] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:50.571194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f80af-5ce1-4bef-be49-a6caf4da57c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:50.582379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f80af-5ce1-4bef-be49-a6caf4da57c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:50.583825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f80af-5ce1-4bef-be49-a6caf4da57c8 service glance] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:50.594477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1046/2087] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:50.665884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfec29-4cf6-4b2c-b501-020b06bf16b3 None None] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:50.666169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfec29-4cf6-4b2c-b501-020b06bf16b3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:50.666456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfec29-4cf6-4b2c-b501-020b06bf16b3 None None] PATH_INFO: `/` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:50.667821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1042/2088] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:11:50.673341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4a55f-af64-4812-9c9f-fb7e9694cbf0 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:50.673634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4a55f-af64-4812-9c9f-fb7e9694cbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:50.673888 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4a55f-af64-4812-9c9f-fb7e9694cbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:50.694241 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-98f4a55f-af64-4812-9c9f-fb7e9694cbf0 None None] MFA Rules not processed for user `9cffb662f4104e349c81c578bf66ab7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:11:50.695538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-98f4a55f-af64-4812-9c9f-fb7e9694cbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:50.701990 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1047/2089] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:11:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:11:50.713761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfec29-4cf6-4b2c-b501-020b06bf16b3 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:50.724601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfec29-4cf6-4b2c-b501-020b06bf16b3 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:50.728108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b0b58-3c58-4e5b-9894-a4160e67acb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:50.728697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b0b58-3c58-4e5b-9894-a4160e67acb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:50.728973 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b0b58-3c58-4e5b-9894-a4160e67acb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:50.729247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b0b58-3c58-4e5b-9894-a4160e67acb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:50.737187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b0b58-3c58-4e5b-9894-a4160e67acb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:50.738613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b0b58-3c58-4e5b-9894-a4160e67acb1 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:50.746304 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1043/2090] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:50.835177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6994d-2da2-47ef-aba2-af60c5ef6640 None None] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:50.835439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6994d-2da2-47ef-aba2-af60c5ef6640 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:50.835727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6994d-2da2-47ef-aba2-af60c5ef6640 None None] PATH_INFO: `/` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:50.837118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1048/2091] 10.222.0.58 () {54 vars in 921 bytes} [Fri Oct 28 19:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 19:11:50.843080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687aae8-8e5f-4515-90ed-e5fda07e26c2 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:50.843364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687aae8-8e5f-4515-90ed-e5fda07e26c2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:50.843670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687aae8-8e5f-4515-90ed-e5fda07e26c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:50.864438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-7687aae8-8e5f-4515-90ed-e5fda07e26c2 None None] MFA Rules not processed for user `a758611cf8d94e8ca8aad6a75bef5e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:11:50.865642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7687aae8-8e5f-4515-90ed-e5fda07e26c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:50.874874 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1044/2092] 10.222.0.58 () {58 vars in 1018 bytes} [Fri Oct 28 19:11:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:11:50.885666 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6994d-2da2-47ef-aba2-af60c5ef6640 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:50.896728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6994d-2da2-47ef-aba2-af60c5ef6640 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:50.900280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff767e04-e594-4962-962a-b77fbc1a9e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:50.900872 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff767e04-e594-4962-962a-b77fbc1a9e5c service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:50.901145 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff767e04-e594-4962-962a-b77fbc1a9e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:50.901413 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff767e04-e594-4962-962a-b77fbc1a9e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:50.912280 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff767e04-e594-4962-962a-b77fbc1a9e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:50.913690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff767e04-e594-4962-962a-b77fbc1a9e5c service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:50.925336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1049/2093] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:11:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:11:53.311840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7733352-9256-4268-be43-ed41c13aa343 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:53.312318 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7733352-9256-4268-be43-ed41c13aa343 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:53.312411 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7733352-9256-4268-be43-ed41c13aa343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:53.332926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d7733352-9256-4268-be43-ed41c13aa343 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:11:53.334125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d7733352-9256-4268-be43-ed41c13aa343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:11:53.341248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1045/2094] 10.222.0.58 () {58 vars in 1009 bytes} [Fri Oct 28 19:11:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:11:53.353178 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff767e04-e594-4962-962a-b77fbc1a9e5c service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:11:53.364105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff767e04-e594-4962-962a-b77fbc1a9e5c service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:11:53.369538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681297d2-66d0-43f5-9b22-eeab55732994 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:11:53.370141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681297d2-66d0-43f5-9b22-eeab55732994 service swift] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:11:53.370419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681297d2-66d0-43f5-9b22-eeab55732994 service swift] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:11:53.370689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681297d2-66d0-43f5-9b22-eeab55732994 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:11:53.379610 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681297d2-66d0-43f5-9b22-eeab55732994 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:11:53.381188 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681297d2-66d0-43f5-9b22-eeab55732994 service swift] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:11:53.389672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1050/2095] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:11:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 19:12:03.326564 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=15 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:03.386307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=41 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:06.863960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7733352-9256-4268-be43-ed41c13aa343 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:06.873115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7733352-9256-4268-be43-ed41c13aa343 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:06.876740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:06.877324 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:06.877597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:06.877861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:06.879549 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:06.941456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:06.942806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:06.952314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1046/2096] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:10.649855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681297d2-66d0-43f5-9b22-eeab55732994 service swift] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:10.651437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-681297d2-66d0-43f5-9b22-eeab55732994 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:10.666291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681297d2-66d0-43f5-9b22-eeab55732994 service swift] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:10.673697 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629dd08-8e39-4ea9-93ed-10962b94eeed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da958657ac3041848d24a76b76705489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:10.674323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629dd08-8e39-4ea9-93ed-10962b94eeed service placement] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:10.674705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629dd08-8e39-4ea9-93ed-10962b94eeed service placement] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:10.674872 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629dd08-8e39-4ea9-93ed-10962b94eeed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:10.676672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-8629dd08-8e39-4ea9-93ed-10962b94eeed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:10.690463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629dd08-8e39-4ea9-93ed-10962b94eeed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:10.692005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629dd08-8e39-4ea9-93ed-10962b94eeed service placement] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:10.705411 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1051/2097] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:10.712481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:10.720299 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:10.737306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c99dc-4b4b-4297-a09f-1e1709486c3d service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:10.743172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:10.743997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:10.744326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:10.744614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] PATH_INFO: `/v3/users/c8ca34fc5c5743d0b5e3f4f97ae285e0` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:10.745821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ca34fc5c5743d0b5e3f4f97ae285e0)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:10.747824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:10.813078 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ca34fc5c5743d0b5e3f4f97ae285e0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:10.813861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ca34fc5c5743d0b5e3f4f97ae285e0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:10.828044 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ca34fc5c5743d0b5e3f4f97ae285e0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:10.835909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ca34fc5c5743d0b5e3f4f97ae285e0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:10.843803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7c26476a-e89e-4c9d-a6c3-95e85e7a5bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ca34fc5c5743d0b5e3f4f97ae285e0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:10.851055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1047/2098] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:10 2022] DELETE /identity/v3/users/c8ca34fc5c5743d0b5e3f4f97ae285e0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:11.087207 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629dd08-8e39-4ea9-93ed-10962b94eeed service placement] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:11.099488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629dd08-8e39-4ea9-93ed-10962b94eeed service placement] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:11.194873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:11.195946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.196398 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.196884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] PATH_INFO: `/v3/projects/4f6c133d9594455182ac1b9ce13df65c` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.198804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f6c133d9594455182ac1b9ce13df65c)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:11.201716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:11.261209 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f6c133d9594455182ac1b9ce13df65c', 'request_id': 'req-72aa5957-d4c3-4c7b-b807-a521b91bd452'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:11.269675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f6c133d9594455182ac1b9ce13df65c', 'request_id': 'req-72aa5957-d4c3-4c7b-b807-a521b91bd452'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:11.282251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f6c133d9594455182ac1b9ce13df65c', 'request_id': 'req-72aa5957-d4c3-4c7b-b807-a521b91bd452'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:11.293585 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] The token cache is being invalidate because project 4f6c133d9594455182ac1b9ce13df65c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:11.293875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:11.294562 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-72aa5957-d4c3-4c7b-b807-a521b91bd452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:11.296077 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1052/2099] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:11 2022] DELETE /identity/v3/projects/4f6c133d9594455182ac1b9ce13df65c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:11.302870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fab6a-46a2-44dc-beae-86c5a81699bc None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.303271 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fab6a-46a2-44dc-beae-86c5a81699bc None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.303557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fab6a-46a2-44dc-beae-86c5a81699bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.328385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-3a7fab6a-46a2-44dc-beae-86c5a81699bc None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:11.330219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3a7fab6a-46a2-44dc-beae-86c5a81699bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:11.400866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1048/2100] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:11.410327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119df8d-6328-4486-9b0a-48df1b9eac6b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.410642 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119df8d-6328-4486-9b0a-48df1b9eac6b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.410920 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119df8d-6328-4486-9b0a-48df1b9eac6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.433002 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-a119df8d-6328-4486-9b0a-48df1b9eac6b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:11.434223 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a119df8d-6328-4486-9b0a-48df1b9eac6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:11.443861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1053/2101] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:11.448260 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fab6a-46a2-44dc-beae-86c5a81699bc None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:11.458888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fab6a-46a2-44dc-beae-86c5a81699bc None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:11.463840 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b60c84-0608-4ac8-a32e-4e865ee89c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:11.464498 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b60c84-0608-4ac8-a32e-4e865ee89c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.464806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b60c84-0608-4ac8-a32e-4e865ee89c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.465081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b60c84-0608-4ac8-a32e-4e865ee89c6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.465732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b60c84-0608-4ac8-a32e-4e865ee89c6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:12:11.466040 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b60c84-0608-4ac8-a32e-4e865ee89c6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:11.467734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b60c84-0608-4ac8-a32e-4e865ee89c6f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:11.470105 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:11.487552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1049/2102] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:12:11.491895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119df8d-6328-4486-9b0a-48df1b9eac6b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:11.501357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119df8d-6328-4486-9b0a-48df1b9eac6b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:11.506679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725d8ea-ec38-40e9-853c-1e5b09738908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:11.507336 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725d8ea-ec38-40e9-853c-1e5b09738908 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.507592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725d8ea-ec38-40e9-853c-1e5b09738908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.507860 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725d8ea-ec38-40e9-853c-1e5b09738908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.508828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725d8ea-ec38-40e9-853c-1e5b09738908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:11.510861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725d8ea-ec38-40e9-853c-1e5b09738908 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:11.541796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1054/2103] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:11 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:11.545912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b60c84-0608-4ac8-a32e-4e865ee89c6f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:11.556041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b60c84-0608-4ac8-a32e-4e865ee89c6f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:11.619354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46e1a7-84a9-494a-8a2c-e8207999e2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:11.619882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46e1a7-84a9-494a-8a2c-e8207999e2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.620159 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46e1a7-84a9-494a-8a2c-e8207999e2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.620432 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46e1a7-84a9-494a-8a2c-e8207999e2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.621071 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46e1a7-84a9-494a-8a2c-e8207999e2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:11.623072 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46e1a7-84a9-494a-8a2c-e8207999e2b1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:11.644997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1050/2104] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:11 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:11.649221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725d8ea-ec38-40e9-853c-1e5b09738908 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:11.660262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725d8ea-ec38-40e9-853c-1e5b09738908 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:11.665001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2d403-6999-44cd-a4f6-ed7240a881db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:11.665600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2d403-6999-44cd-a4f6-ed7240a881db admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.665873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2d403-6999-44cd-a4f6-ed7240a881db admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.666138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2d403-6999-44cd-a4f6-ed7240a881db admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.666775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2d403-6999-44cd-a4f6-ed7240a881db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:11.668450 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2d403-6999-44cd-a4f6-ed7240a881db admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:11.679589 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1055/2105] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:11.683809 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46e1a7-84a9-494a-8a2c-e8207999e2b1 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:11.695320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46e1a7-84a9-494a-8a2c-e8207999e2b1 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:11.703005 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe846318-1b59-4516-8c2f-df783fa91cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:11.703609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe846318-1b59-4516-8c2f-df783fa91cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.704017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe846318-1b59-4516-8c2f-df783fa91cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.704220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe846318-1b59-4516-8c2f-df783fa91cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.704876 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe846318-1b59-4516-8c2f-df783fa91cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:11.706620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe846318-1b59-4516-8c2f-df783fa91cc0 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:11.728819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1051/2106] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:11.731527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2d403-6999-44cd-a4f6-ed7240a881db admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:11.742641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a38df-ec24-4ad1-aef6-ecdb5d4c5913 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.743203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a38df-ec24-4ad1-aef6-ecdb5d4c5913 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.743203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a38df-ec24-4ad1-aef6-ecdb5d4c5913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.750638 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2d403-6999-44cd-a4f6-ed7240a881db admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:11.762858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1b8ae-8b9a-4b6e-ae03-d64e2f3e775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:11.763945 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1b8ae-8b9a-4b6e-ae03-d64e2f3e775a admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.764506 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1b8ae-8b9a-4b6e-ae03-d64e2f3e775a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.764849 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1b8ae-8b9a-4b6e-ae03-d64e2f3e775a admin admin] PATH_INFO: `/v3/projects/b5a1b2316802416e9a2cc021ce2c4dc1/users/3cbc160f7ea04086902e47299a83eb03/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.775545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f06a38df-ec24-4ad1-aef6-ecdb5d4c5913 None None] MFA Rules not processed for user `0eef740ccf884c079e6a2d6cdc89f4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:11.777028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f06a38df-ec24-4ad1-aef6-ecdb5d4c5913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:11.783265 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1b8ae-8b9a-4b6e-ae03-d64e2f3e775a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a1b2316802416e9a2cc021ce2c4dc1, user_id=3cbc160f7ea04086902e47299a83eb03, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:11.787204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1b8ae-8b9a-4b6e-ae03-d64e2f3e775a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:11.804623 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1056/2107] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:11 2022] PUT /identity/v3/projects/b5a1b2316802416e9a2cc021ce2c4dc1/users/3cbc160f7ea04086902e47299a83eb03/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:11.817041 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1ef92-c963-4992-a2a7-e6ebdd4cc3f5 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:11.817692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1ef92-c963-4992-a2a7-e6ebdd4cc3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:11.818091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1ef92-c963-4992-a2a7-e6ebdd4cc3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:11.845259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:11.849733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-2bc1ef92-c963-4992-a2a7-e6ebdd4cc3f5 None None] MFA Rules not processed for user `3cbc160f7ea04086902e47299a83eb03`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:11.851374 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1052/2108] 10.222.0.58 () {58 vars in 967 bytes} [Fri Oct 28 19:12:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:11.851542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2bc1ef92-c963-4992-a2a7-e6ebdd4cc3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:11.865713 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a38df-ec24-4ad1-aef6-ecdb5d4c5913 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:11.867164 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f06a38df-ec24-4ad1-aef6-ecdb5d4c5913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:11.925491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1057/2109] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:11.936120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1ef92-c963-4992-a2a7-e6ebdd4cc3f5 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:11.938012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-2bc1ef92-c963-4992-a2a7-e6ebdd4cc3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:11.943253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a38df-ec24-4ad1-aef6-ecdb5d4c5913 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:11.962137 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:12.009155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564b859-1a14-4b0c-816e-2a3df4bac93b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a51690d0367b45db8a6552764cc1d11e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:12.009870 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564b859-1a14-4b0c-816e-2a3df4bac93b service swift] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:12.010223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564b859-1a14-4b0c-816e-2a3df4bac93b service swift] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:12.010570 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564b859-1a14-4b0c-816e-2a3df4bac93b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:12.021635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1ef92-c963-4992-a2a7-e6ebdd4cc3f5 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:12.028026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759a26-efa5-4212-b104-ba7e1c5f2aef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:12.028639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759a26-efa5-4212-b104-ba7e1c5f2aef service nova] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:12.028926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759a26-efa5-4212-b104-ba7e1c5f2aef service nova] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:12.029198 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759a26-efa5-4212-b104-ba7e1c5f2aef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:12.037588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759a26-efa5-4212-b104-ba7e1c5f2aef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:12.039212 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759a26-efa5-4212-b104-ba7e1c5f2aef service nova] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:12.048979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1058/2110] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:12:12.080301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564b859-1a14-4b0c-816e-2a3df4bac93b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:12.082237 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564b859-1a14-4b0c-816e-2a3df4bac93b service swift] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:12.091183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1053/2111] 10.222.0.58 () {60 vars in 1384 bytes} [Fri Oct 28 19:12:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 28 19:12:19.491204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759a26-efa5-4212-b104-ba7e1c5f2aef service nova] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:19.554448 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759a26-efa5-4212-b104-ba7e1c5f2aef service nova] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:19.559079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:19.559703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:19.559974 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:19.560241 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] PATH_INFO: `/v3/users/3cbc160f7ea04086902e47299a83eb03` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:19.561340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cbc160f7ea04086902e47299a83eb03)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:19.563269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:19.598843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:19.615956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cbc160f7ea04086902e47299a83eb03'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:19.616518 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cbc160f7ea04086902e47299a83eb03'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:19.629052 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cbc160f7ea04086902e47299a83eb03'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:19.636135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cbc160f7ea04086902e47299a83eb03'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:19.636863 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cbc160f7ea04086902e47299a83eb03'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:19.650080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1059/2112] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:19 2022] DELETE /identity/v3/users/3cbc160f7ea04086902e47299a83eb03 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:19.654638 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464e3c-317a-42a3-a4ba-b587bb6a3471 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:19.655152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464e3c-317a-42a3-a4ba-b587bb6a3471 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:19.655298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464e3c-317a-42a3-a4ba-b587bb6a3471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:19.675340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-25464e3c-317a-42a3-a4ba-b587bb6a3471 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:19.676541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-25464e3c-317a-42a3-a4ba-b587bb6a3471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:19.738296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1054/2113] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:19.748385 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:19.749996 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:19.811371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaec789-0ebe-44aa-a627-1b7d99ec7565 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:19.815664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e4d65-b060-4f0f-afb6-ac541ea43a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:19.816271 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e4d65-b060-4f0f-afb6-ac541ea43a0d service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:19.816549 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e4d65-b060-4f0f-afb6-ac541ea43a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:19.816819 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e4d65-b060-4f0f-afb6-ac541ea43a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:19.826221 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e4d65-b060-4f0f-afb6-ac541ea43a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:19.827762 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e4d65-b060-4f0f-afb6-ac541ea43a0d service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:19.837632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1060/2114] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:20.120614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25464e3c-317a-42a3-a4ba-b587bb6a3471 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:20.122472 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-25464e3c-317a-42a3-a4ba-b587bb6a3471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:20.137170 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25464e3c-317a-42a3-a4ba-b587bb6a3471 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:20.143464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:20.144538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.144838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.145220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] PATH_INFO: `/v3/projects/b5a1b2316802416e9a2cc021ce2c4dc1` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.146397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5a1b2316802416e9a2cc021ce2c4dc1)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:20.148697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:20.198269 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5a1b2316802416e9a2cc021ce2c4dc1', 'request_id': 'req-7e659f0f-8adc-4328-a924-4ea72227922c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:20.205406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5a1b2316802416e9a2cc021ce2c4dc1', 'request_id': 'req-7e659f0f-8adc-4328-a924-4ea72227922c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:20.208052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:20.213797 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5a1b2316802416e9a2cc021ce2c4dc1', 'request_id': 'req-7e659f0f-8adc-4328-a924-4ea72227922c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:20.225222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] The token cache is being invalidate because project b5a1b2316802416e9a2cc021ce2c4dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:20.225507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:20.226140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7e659f0f-8adc-4328-a924-4ea72227922c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:20.227668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1055/2115] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:20 2022] DELETE /identity/v3/projects/b5a1b2316802416e9a2cc021ce2c4dc1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:20.232834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73973d6a-f4a9-4a38-9cc3-663476f2da03 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.233528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73973d6a-f4a9-4a38-9cc3-663476f2da03 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.233674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73973d6a-f4a9-4a38-9cc3-663476f2da03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.254312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-73973d6a-f4a9-4a38-9cc3-663476f2da03 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:20.255573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-73973d6a-f4a9-4a38-9cc3-663476f2da03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:20.319718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1061/2116] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:20.327655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376c1be-9f21-44cd-bcc7-1ad1ef270251 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.327941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376c1be-9f21-44cd-bcc7-1ad1ef270251 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.328211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376c1be-9f21-44cd-bcc7-1ad1ef270251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.348848 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-7376c1be-9f21-44cd-bcc7-1ad1ef270251 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:20.350058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7376c1be-9f21-44cd-bcc7-1ad1ef270251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:20.358176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1056/2117] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:20.362505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73973d6a-f4a9-4a38-9cc3-663476f2da03 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:20.380598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73973d6a-f4a9-4a38-9cc3-663476f2da03 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:20.387566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283d6e3-d2a6-4f27-bc1e-904e28c79805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:20.388382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283d6e3-d2a6-4f27-bc1e-904e28c79805 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.388761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283d6e3-d2a6-4f27-bc1e-904e28c79805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.389132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283d6e3-d2a6-4f27-bc1e-904e28c79805 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.390053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283d6e3-d2a6-4f27-bc1e-904e28c79805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:12:20.390457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283d6e3-d2a6-4f27-bc1e-904e28c79805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:20.393403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283d6e3-d2a6-4f27-bc1e-904e28c79805 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:20.411042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1062/2118] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:12:20.416569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376c1be-9f21-44cd-bcc7-1ad1ef270251 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:20.428947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376c1be-9f21-44cd-bcc7-1ad1ef270251 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:20.436139 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549b20f-da30-4f92-9aaf-e3f4542b0c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:20.436951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549b20f-da30-4f92-9aaf-e3f4542b0c05 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.437448 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549b20f-da30-4f92-9aaf-e3f4542b0c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.438303 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549b20f-da30-4f92-9aaf-e3f4542b0c05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.439283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549b20f-da30-4f92-9aaf-e3f4542b0c05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:20.441751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549b20f-da30-4f92-9aaf-e3f4542b0c05 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:20.476737 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1057/2119] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:20 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:20.481927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283d6e3-d2a6-4f27-bc1e-904e28c79805 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:20.495193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283d6e3-d2a6-4f27-bc1e-904e28c79805 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:20.537292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:20.570046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607647c-15cd-41a6-8ee9-8786eebcdd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:20.570939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607647c-15cd-41a6-8ee9-8786eebcdd3b admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.571348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607647c-15cd-41a6-8ee9-8786eebcdd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.571707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607647c-15cd-41a6-8ee9-8786eebcdd3b admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.572636 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607647c-15cd-41a6-8ee9-8786eebcdd3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:20.575501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607647c-15cd-41a6-8ee9-8786eebcdd3b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:20.606445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1063/2120] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:20 2022] POST /identity/v3/users => generated 303 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:20.612118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549b20f-da30-4f92-9aaf-e3f4542b0c05 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:20.625313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549b20f-da30-4f92-9aaf-e3f4542b0c05 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:20.631670 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f3226-79d6-441e-ac13-2d35398d1679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:20.632227 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f3226-79d6-441e-ac13-2d35398d1679 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.633002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f3226-79d6-441e-ac13-2d35398d1679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.633002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f3226-79d6-441e-ac13-2d35398d1679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.633650 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f3226-79d6-441e-ac13-2d35398d1679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:20.635356 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f3226-79d6-441e-ac13-2d35398d1679 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:20.647557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1058/2121] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:20.652119 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607647c-15cd-41a6-8ee9-8786eebcdd3b admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:20.664083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607647c-15cd-41a6-8ee9-8786eebcdd3b admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:20.670054 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41e6ba-cbe6-469f-bfc2-44930d6430b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:20.670763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41e6ba-cbe6-469f-bfc2-44930d6430b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.671132 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41e6ba-cbe6-469f-bfc2-44930d6430b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.671407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41e6ba-cbe6-469f-bfc2-44930d6430b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.672660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41e6ba-cbe6-469f-bfc2-44930d6430b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:20.674496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41e6ba-cbe6-469f-bfc2-44930d6430b3 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:20.687406 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1064/2122] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:20.692513 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f3226-79d6-441e-ac13-2d35398d1679 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:20.711559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f3226-79d6-441e-ac13-2d35398d1679 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:20.719847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3d59b-f88c-4ac2-a961-2d5d2d536375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:20.721027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3d59b-f88c-4ac2-a961-2d5d2d536375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.721496 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3d59b-f88c-4ac2-a961-2d5d2d536375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.722008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3d59b-f88c-4ac2-a961-2d5d2d536375 admin admin] PATH_INFO: `/v3/projects/97f035c0ca0e4c0da9b2c73041e379e3/users/12378fe054334b028dc0c4bd53f486f0/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.746434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3d59b-f88c-4ac2-a961-2d5d2d536375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f035c0ca0e4c0da9b2c73041e379e3, user_id=12378fe054334b028dc0c4bd53f486f0, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:20.749617 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3d59b-f88c-4ac2-a961-2d5d2d536375 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:20.769036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1059/2123] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:20 2022] PUT /identity/v3/projects/97f035c0ca0e4c0da9b2c73041e379e3/users/12378fe054334b028dc0c4bd53f486f0/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:20.780902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b4cb0-0d39-4f91-9df9-a9a6e8b5b143 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.781228 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b4cb0-0d39-4f91-9df9-a9a6e8b5b143 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.781522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b4cb0-0d39-4f91-9df9-a9a6e8b5b143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.811968 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-687b4cb0-0d39-4f91-9df9-a9a6e8b5b143 None None] MFA Rules not processed for user `12378fe054334b028dc0c4bd53f486f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:20.815199 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-687b4cb0-0d39-4f91-9df9-a9a6e8b5b143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:20.899606 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1065/2124] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:20.906115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3d59b-f88c-4ac2-a961-2d5d2d536375 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:20.984724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3d59b-f88c-4ac2-a961-2d5d2d536375 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:20.991291 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e187c1-b6e6-45e9-9594-093b545a7d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:20.992375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e187c1-b6e6-45e9-9594-093b545a7d2c admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:20.992375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e187c1-b6e6-45e9-9594-093b545a7d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:20.992375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e187c1-b6e6-45e9-9594-093b545a7d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:20.992375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e187c1-b6e6-45e9-9594-093b545a7d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:20.995688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e187c1-b6e6-45e9-9594-093b545a7d2c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:21.024443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1060/2125] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:20 2022] POST /identity/v3/projects => generated 405 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:21.028518 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b4cb0-0d39-4f91-9df9-a9a6e8b5b143 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:21.039837 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b4cb0-0d39-4f91-9df9-a9a6e8b5b143 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:21.102070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf08df-cc6e-4388-8940-21f2c4c1499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:21.102799 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf08df-cc6e-4388-8940-21f2c4c1499a admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:21.103152 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf08df-cc6e-4388-8940-21f2c4c1499a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:21.103423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf08df-cc6e-4388-8940-21f2c4c1499a admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:21.104068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf08df-cc6e-4388-8940-21f2c4c1499a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:21.105995 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf08df-cc6e-4388-8940-21f2c4c1499a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:21.132670 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1066/2126] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:21 2022] POST /identity/v3/users => generated 309 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:21.135542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e187c1-b6e6-45e9-9594-093b545a7d2c admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:21.149573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e187c1-b6e6-45e9-9594-093b545a7d2c admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:21.155353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a2080-f8ea-4213-a40c-de59550e0599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:21.155952 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a2080-f8ea-4213-a40c-de59550e0599 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:21.156249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a2080-f8ea-4213-a40c-de59550e0599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:21.156582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a2080-f8ea-4213-a40c-de59550e0599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:21.157489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a2080-f8ea-4213-a40c-de59550e0599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:21.160382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a2080-f8ea-4213-a40c-de59550e0599 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:21.174005 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1061/2127] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:12:21.178649 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf08df-cc6e-4388-8940-21f2c4c1499a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:21.191815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf08df-cc6e-4388-8940-21f2c4c1499a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:21.198587 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0fe4b-5b46-45f4-81b6-af94f442f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:21.199595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0fe4b-5b46-45f4-81b6-af94f442f709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:21.199940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0fe4b-5b46-45f4-81b6-af94f442f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:21.200317 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0fe4b-5b46-45f4-81b6-af94f442f709 admin admin] PATH_INFO: `/v3/projects/13a9331dda8243aab7763d1f1a5b738f/users/aec73fcc504846ab8d598532188e0614/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:21.221059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0fe4b-5b46-45f4-81b6-af94f442f709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a9331dda8243aab7763d1f1a5b738f, user_id=aec73fcc504846ab8d598532188e0614, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:21.224102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0fe4b-5b46-45f4-81b6-af94f442f709 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:21.240263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1067/2128] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:21 2022] PUT /identity/v3/projects/13a9331dda8243aab7763d1f1a5b738f/users/aec73fcc504846ab8d598532188e0614/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:21.244397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a2080-f8ea-4213-a40c-de59550e0599 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:21.256829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a2080-f8ea-4213-a40c-de59550e0599 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:21.324735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:21.336470 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd7025-11c5-49b6-9239-6108915bf01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:21.337283 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd7025-11c5-49b6-9239-6108915bf01f admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:21.337835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd7025-11c5-49b6-9239-6108915bf01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:21.338160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd7025-11c5-49b6-9239-6108915bf01f admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:21.339885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd7025-11c5-49b6-9239-6108915bf01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:21.341898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd7025-11c5-49b6-9239-6108915bf01f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:21.356479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1062/2129] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:21.361364 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0fe4b-5b46-45f4-81b6-af94f442f709 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:21.373047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0fe4b-5b46-45f4-81b6-af94f442f709 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:21.378257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208bd2e-12ad-4751-a945-2141ba3adec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:21.379014 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208bd2e-12ad-4751-a945-2141ba3adec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:21.379323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208bd2e-12ad-4751-a945-2141ba3adec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:21.379632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208bd2e-12ad-4751-a945-2141ba3adec1 admin admin] PATH_INFO: `/v3/projects/13a9331dda8243aab7763d1f1a5b738f/users/aec73fcc504846ab8d598532188e0614/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:21.381762 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208bd2e-12ad-4751-a945-2141ba3adec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a9331dda8243aab7763d1f1a5b738f, user_id=aec73fcc504846ab8d598532188e0614, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:21.385174 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208bd2e-12ad-4751-a945-2141ba3adec1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:21.393201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:21.400182 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1068/2130] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:21 2022] PUT /identity/v3/projects/13a9331dda8243aab7763d1f1a5b738f/users/aec73fcc504846ab8d598532188e0614/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:21.412397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0a142-edc6-4ffa-9e79-35666eabd2bf None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:21.412687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0a142-edc6-4ffa-9e79-35666eabd2bf None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:21.413028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0a142-edc6-4ffa-9e79-35666eabd2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:21.447996 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-0fb0a142-edc6-4ffa-9e79-35666eabd2bf None None] MFA Rules not processed for user `aec73fcc504846ab8d598532188e0614`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:21.449163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0fb0a142-edc6-4ffa-9e79-35666eabd2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:21.519734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1063/2131] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:21 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:21.532894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208bd2e-12ad-4751-a945-2141ba3adec1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:21.534796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-7208bd2e-12ad-4751-a945-2141ba3adec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:21.600602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208bd2e-12ad-4751-a945-2141ba3adec1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:21.604594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52736d26-fa2d-47c5-b036-87e1b4efc875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:21.605203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52736d26-fa2d-47c5-b036-87e1b4efc875 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:21.605480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52736d26-fa2d-47c5-b036-87e1b4efc875 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:21.605751 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52736d26-fa2d-47c5-b036-87e1b4efc875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:21.615874 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52736d26-fa2d-47c5-b036-87e1b4efc875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:21.617311 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52736d26-fa2d-47c5-b036-87e1b4efc875 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:21.626924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1069/2132] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:21 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:21.800806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0a142-edc6-4ffa-9e79-35666eabd2bf None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:21.871734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0a142-edc6-4ffa-9e79-35666eabd2bf None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:21.876556 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:21.877189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:21.877466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:21.877733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] PATH_INFO: `/v3/users/12378fe054334b028dc0c4bd53f486f0` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:21.878863 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12378fe054334b028dc0c4bd53f486f0)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:21.880847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:21.932089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12378fe054334b028dc0c4bd53f486f0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:21.932740 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12378fe054334b028dc0c4bd53f486f0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:21.947381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12378fe054334b028dc0c4bd53f486f0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:21.955682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12378fe054334b028dc0c4bd53f486f0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:21.960011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:21.964183 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12378fe054334b028dc0c4bd53f486f0'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:21.974352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1064/2133] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:21 2022] DELETE /identity/v3/users/12378fe054334b028dc0c4bd53f486f0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:21.980217 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78722804-2526-4df8-94fb-c5d487bbb5af None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:21.980591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78722804-2526-4df8-94fb-c5d487bbb5af None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:21.980946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78722804-2526-4df8-94fb-c5d487bbb5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:22.003356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-78722804-2526-4df8-94fb-c5d487bbb5af None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:22.004818 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-78722804-2526-4df8-94fb-c5d487bbb5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:22.071262 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1070/2134] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:22.081466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:22.082922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:22.150693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daedcc-e9af-4a7e-9ceb-72402eb22e73 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:22.155082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f56b30-1d42-40aa-8c11-bbe6c3836a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:22.155778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f56b30-1d42-40aa-8c11-bbe6c3836a3c service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:22.156130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f56b30-1d42-40aa-8c11-bbe6c3836a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:22.156507 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f56b30-1d42-40aa-8c11-bbe6c3836a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:22.167499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f56b30-1d42-40aa-8c11-bbe6c3836a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:22.169137 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f56b30-1d42-40aa-8c11-bbe6c3836a3c service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:22.179436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1065/2135] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:22.469399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78722804-2526-4df8-94fb-c5d487bbb5af None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:22.471723 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-78722804-2526-4df8-94fb-c5d487bbb5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:22.486998 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78722804-2526-4df8-94fb-c5d487bbb5af None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:22.492610 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:22.493458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:22.493826 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:22.494167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] PATH_INFO: `/v3/projects/97f035c0ca0e4c0da9b2c73041e379e3` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:22.495475 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f035c0ca0e4c0da9b2c73041e379e3)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:22.498163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:22.551586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f035c0ca0e4c0da9b2c73041e379e3', 'request_id': 'req-14b3f6df-9b5a-484e-8ef2-88628c2aba23'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.560392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f035c0ca0e4c0da9b2c73041e379e3', 'request_id': 'req-14b3f6df-9b5a-484e-8ef2-88628c2aba23'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.567140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f035c0ca0e4c0da9b2c73041e379e3', 'request_id': 'req-14b3f6df-9b5a-484e-8ef2-88628c2aba23'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.577515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] The token cache is being invalidate because project 97f035c0ca0e4c0da9b2c73041e379e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:22.577802 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.578633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.580182 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1071/2136] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:22 2022] DELETE /identity/v3/projects/97f035c0ca0e4c0da9b2c73041e379e3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:12:22.584446 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f56b30-1d42-40aa-8c11-bbe6c3836a3c service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:22.595986 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f56b30-1d42-40aa-8c11-bbe6c3836a3c service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:22.671084 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:22.671878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:22.672892 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:22.673204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] PATH_INFO: `/v3/users/aec73fcc504846ab8d598532188e0614` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:22.675169 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec73fcc504846ab8d598532188e0614)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:22.677666 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:22.731349 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec73fcc504846ab8d598532188e0614'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.732098 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec73fcc504846ab8d598532188e0614'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.745701 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec73fcc504846ab8d598532188e0614'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.753609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec73fcc504846ab8d598532188e0614'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.763425 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-16daff7a-ecc1-4142-a225-39124afb90ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec73fcc504846ab8d598532188e0614'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:22.772486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1066/2137] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:22 2022] DELETE /identity/v3/users/aec73fcc504846ab8d598532188e0614 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:23.091823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:23.093861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:23.108842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3f6df-9b5a-484e-8ef2-88628c2aba23 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:23.135698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:23.186057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:23.186782 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.187126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.187460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] PATH_INFO: `/v3/projects/13a9331dda8243aab7763d1f1a5b738f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.189345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a9331dda8243aab7763d1f1a5b738f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:23.191621 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:23.246056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a9331dda8243aab7763d1f1a5b738f', 'request_id': 'req-d77e508a-7dfb-459a-90ea-f13791ad1113'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:23.255434 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a9331dda8243aab7763d1f1a5b738f', 'request_id': 'req-d77e508a-7dfb-459a-90ea-f13791ad1113'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:23.263331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a9331dda8243aab7763d1f1a5b738f', 'request_id': 'req-d77e508a-7dfb-459a-90ea-f13791ad1113'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:23.275172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] The token cache is being invalidate because project 13a9331dda8243aab7763d1f1a5b738f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:23.275457 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:23.276120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-d77e508a-7dfb-459a-90ea-f13791ad1113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:23.277828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1072/2138] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:23 2022] DELETE /identity/v3/projects/13a9331dda8243aab7763d1f1a5b738f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:12:23.283108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58551b96-00c6-437d-b44d-5737c8524713 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.283770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58551b96-00c6-437d-b44d-5737c8524713 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.283770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58551b96-00c6-437d-b44d-5737c8524713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.308930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-58551b96-00c6-437d-b44d-5737c8524713 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:23.310335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-58551b96-00c6-437d-b44d-5737c8524713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:23.325357 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:23.381447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1067/2139] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:23.393136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d391cb-ceed-4789-81ac-c1d974335340 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.393550 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d391cb-ceed-4789-81ac-c1d974335340 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.393942 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d391cb-ceed-4789-81ac-c1d974335340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.421325 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e8d391cb-ceed-4789-81ac-c1d974335340 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:23.423044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e8d391cb-ceed-4789-81ac-c1d974335340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:23.434046 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1073/2140] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:23.439021 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58551b96-00c6-437d-b44d-5737c8524713 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:23.448930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58551b96-00c6-437d-b44d-5737c8524713 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:23.455008 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a55d1-9b22-4441-8f90-2fbc1b0012c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:23.455640 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a55d1-9b22-4441-8f90-2fbc1b0012c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.455921 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a55d1-9b22-4441-8f90-2fbc1b0012c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.456194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a55d1-9b22-4441-8f90-2fbc1b0012c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.456872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a55d1-9b22-4441-8f90-2fbc1b0012c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:12:23.457172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a55d1-9b22-4441-8f90-2fbc1b0012c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:23.458816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a55d1-9b22-4441-8f90-2fbc1b0012c4 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:23.471747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1068/2141] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:12:23.600816 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d391cb-ceed-4789-81ac-c1d974335340 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:23.612724 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d391cb-ceed-4789-81ac-c1d974335340 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:23.619583 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96881ac-f883-4156-a5f2-a71081aa6574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:23.620274 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96881ac-f883-4156-a5f2-a71081aa6574 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.620519 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96881ac-f883-4156-a5f2-a71081aa6574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.620784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96881ac-f883-4156-a5f2-a71081aa6574 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.621494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96881ac-f883-4156-a5f2-a71081aa6574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:23.623567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96881ac-f883-4156-a5f2-a71081aa6574 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:23.661685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1074/2142] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:23 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:12:23.666772 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a55d1-9b22-4441-8f90-2fbc1b0012c4 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:23.678091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a55d1-9b22-4441-8f90-2fbc1b0012c4 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:23.754891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff63d2-b7da-4e35-8143-45224f5d2ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:23.755545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff63d2-b7da-4e35-8143-45224f5d2ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.755826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff63d2-b7da-4e35-8143-45224f5d2ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.756093 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff63d2-b7da-4e35-8143-45224f5d2ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.756732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff63d2-b7da-4e35-8143-45224f5d2ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:23.758924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff63d2-b7da-4e35-8143-45224f5d2ce9 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:23.782913 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1069/2143] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:23 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:23.787592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96881ac-f883-4156-a5f2-a71081aa6574 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:23.799804 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96881ac-f883-4156-a5f2-a71081aa6574 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:23.805186 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e38e6-d8b6-427e-97c8-8e8a9c3e89d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:23.805849 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e38e6-d8b6-427e-97c8-8e8a9c3e89d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.806185 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e38e6-d8b6-427e-97c8-8e8a9c3e89d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.806529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e38e6-d8b6-427e-97c8-8e8a9c3e89d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.807250 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e38e6-d8b6-427e-97c8-8e8a9c3e89d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:23.809213 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e38e6-d8b6-427e-97c8-8e8a9c3e89d7 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:23.821093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1075/2144] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:23.825561 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff63d2-b7da-4e35-8143-45224f5d2ce9 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:23.838085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff63d2-b7da-4e35-8143-45224f5d2ce9 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:23.842858 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bcf6c-0fd4-4e9d-8a6c-bc8879dd102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:23.843519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bcf6c-0fd4-4e9d-8a6c-bc8879dd102c admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.843795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bcf6c-0fd4-4e9d-8a6c-bc8879dd102c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.844353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bcf6c-0fd4-4e9d-8a6c-bc8879dd102c admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.844761 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bcf6c-0fd4-4e9d-8a6c-bc8879dd102c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:23.847270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bcf6c-0fd4-4e9d-8a6c-bc8879dd102c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:23.862007 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1070/2145] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:23.866602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e38e6-d8b6-427e-97c8-8e8a9c3e89d7 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:23.877427 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e38e6-d8b6-427e-97c8-8e8a9c3e89d7 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:23.882313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb647e24-b6d8-4190-b96d-5c598fdf3e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:23.883012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb647e24-b6d8-4190-b96d-5c598fdf3e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.883534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb647e24-b6d8-4190-b96d-5c598fdf3e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.883784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb647e24-b6d8-4190-b96d-5c598fdf3e2d admin admin] PATH_INFO: `/v3/projects/3fd281aef23d49dab9f190c59726d1cd/users/c4668b1cb13b4d4bb64196fe301c91be/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.900258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb647e24-b6d8-4190-b96d-5c598fdf3e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd281aef23d49dab9f190c59726d1cd, user_id=c4668b1cb13b4d4bb64196fe301c91be, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:23.902966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb647e24-b6d8-4190-b96d-5c598fdf3e2d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:23.919082 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1076/2146] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:23 2022] PUT /identity/v3/projects/3fd281aef23d49dab9f190c59726d1cd/users/c4668b1cb13b4d4bb64196fe301c91be/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:12:23.932830 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a141c12-ff55-485c-9932-7ab5940df1cf None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:23.933205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a141c12-ff55-485c-9932-7ab5940df1cf None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:23.933492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a141c12-ff55-485c-9932-7ab5940df1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:23.968703 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-6a141c12-ff55-485c-9932-7ab5940df1cf None None] MFA Rules not processed for user `c4668b1cb13b4d4bb64196fe301c91be`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:23.970749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6a141c12-ff55-485c-9932-7ab5940df1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:24.042592 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1071/2147] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:24.047912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb647e24-b6d8-4190-b96d-5c598fdf3e2d admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:24.098572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:24.112592 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb647e24-b6d8-4190-b96d-5c598fdf3e2d admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:24.117808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba5ce0-947c-4cac-9e07-c7a3418341e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:24.118447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba5ce0-947c-4cac-9e07-c7a3418341e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:24.118733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba5ce0-947c-4cac-9e07-c7a3418341e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:24.119023 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba5ce0-947c-4cac-9e07-c7a3418341e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:24.119656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba5ce0-947c-4cac-9e07-c7a3418341e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:24.121569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba5ce0-947c-4cac-9e07-c7a3418341e2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:24.154050 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1077/2148] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:24 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:24.164387 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a141c12-ff55-485c-9932-7ab5940df1cf None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:24.179544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a141c12-ff55-485c-9932-7ab5940df1cf None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:24.190944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:24.254893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01d6e2-4de9-4c61-8795-018012519910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:24.255656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01d6e2-4de9-4c61-8795-018012519910 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:24.256005 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01d6e2-4de9-4c61-8795-018012519910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:24.256400 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01d6e2-4de9-4c61-8795-018012519910 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:24.257212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01d6e2-4de9-4c61-8795-018012519910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:24.260068 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01d6e2-4de9-4c61-8795-018012519910 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:24.287973 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1072/2149] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:24 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:24.292594 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba5ce0-947c-4cac-9e07-c7a3418341e2 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:24.305069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba5ce0-947c-4cac-9e07-c7a3418341e2 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:24.310824 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a548ff4-1ba5-4522-841c-be08a1605882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:24.311894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a548ff4-1ba5-4522-841c-be08a1605882 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:24.312383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a548ff4-1ba5-4522-841c-be08a1605882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:24.312859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a548ff4-1ba5-4522-841c-be08a1605882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:24.313746 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a548ff4-1ba5-4522-841c-be08a1605882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:24.316435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a548ff4-1ba5-4522-841c-be08a1605882 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:24.334718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1078/2150] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:24.340396 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01d6e2-4de9-4c61-8795-018012519910 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:24.351967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01d6e2-4de9-4c61-8795-018012519910 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:24.357511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4d0b6-2611-40f6-878f-d4f94445b316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:24.358220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4d0b6-2611-40f6-878f-d4f94445b316 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:24.358526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4d0b6-2611-40f6-878f-d4f94445b316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:24.358795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4d0b6-2611-40f6-878f-d4f94445b316 admin admin] PATH_INFO: `/v3/projects/ecadeeb6649545e5928fef10e206d959/users/6ecddcd886f04843a8344f1c8bf5eaf5/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:24.376535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4d0b6-2611-40f6-878f-d4f94445b316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecadeeb6649545e5928fef10e206d959, user_id=6ecddcd886f04843a8344f1c8bf5eaf5, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:24.379366 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4d0b6-2611-40f6-878f-d4f94445b316 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:24.395505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1073/2151] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:24 2022] PUT /identity/v3/projects/ecadeeb6649545e5928fef10e206d959/users/6ecddcd886f04843a8344f1c8bf5eaf5/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:12:24.400095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a548ff4-1ba5-4522-841c-be08a1605882 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:24.413482 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a548ff4-1ba5-4522-841c-be08a1605882 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:24.482579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01159567-cba9-48c7-a68d-dc61acd819f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:24.483329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01159567-cba9-48c7-a68d-dc61acd819f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:24.483657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01159567-cba9-48c7-a68d-dc61acd819f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:24.483979 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01159567-cba9-48c7-a68d-dc61acd819f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:24.484884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01159567-cba9-48c7-a68d-dc61acd819f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:24.486693 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01159567-cba9-48c7-a68d-dc61acd819f8 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:24.500492 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1079/2152] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:24.505785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4d0b6-2611-40f6-878f-d4f94445b316 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:24.519557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4d0b6-2611-40f6-878f-d4f94445b316 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:24.525144 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bddb9b-b953-43b0-bb28-e2ed5ce4f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:24.525929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bddb9b-b953-43b0-bb28-e2ed5ce4f713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:24.526252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bddb9b-b953-43b0-bb28-e2ed5ce4f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:24.526596 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bddb9b-b953-43b0-bb28-e2ed5ce4f713 admin admin] PATH_INFO: `/v3/projects/ecadeeb6649545e5928fef10e206d959/users/6ecddcd886f04843a8344f1c8bf5eaf5/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:24.529207 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bddb9b-b953-43b0-bb28-e2ed5ce4f713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecadeeb6649545e5928fef10e206d959, user_id=6ecddcd886f04843a8344f1c8bf5eaf5, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:24.533058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bddb9b-b953-43b0-bb28-e2ed5ce4f713 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:24.545263 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1074/2153] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:24 2022] PUT /identity/v3/projects/ecadeeb6649545e5928fef10e206d959/users/6ecddcd886f04843a8344f1c8bf5eaf5/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:24.557015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4825598-412e-43b6-b55a-0aa66160064f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:24.557295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4825598-412e-43b6-b55a-0aa66160064f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:24.557565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4825598-412e-43b6-b55a-0aa66160064f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:24.586469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-c4825598-412e-43b6-b55a-0aa66160064f None None] MFA Rules not processed for user `6ecddcd886f04843a8344f1c8bf5eaf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:24.587733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c4825598-412e-43b6-b55a-0aa66160064f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:24.663836 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1080/2154] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:24.675690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bddb9b-b953-43b0-bb28-e2ed5ce4f713 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:24.677530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f2bddb9b-b953-43b0-bb28-e2ed5ce4f713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:24.743847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bddb9b-b953-43b0-bb28-e2ed5ce4f713 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:24.747551 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba9a63-7cb5-4131-a044-da84a33717c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:24.748158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba9a63-7cb5-4131-a044-da84a33717c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:24.748434 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba9a63-7cb5-4131-a044-da84a33717c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:24.748704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba9a63-7cb5-4131-a044-da84a33717c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:24.760636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba9a63-7cb5-4131-a044-da84a33717c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:24.762106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba9a63-7cb5-4131-a044-da84a33717c0 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:24.773117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1075/2155] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:25.235914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4825598-412e-43b6-b55a-0aa66160064f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:25.294001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:25.318817 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4825598-412e-43b6-b55a-0aa66160064f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:25.324669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:25.325654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:25.326043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:25.326598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] PATH_INFO: `/v3/users/c4668b1cb13b4d4bb64196fe301c91be` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:25.328295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4668b1cb13b4d4bb64196fe301c91be)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:25.331712 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:25.396927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4668b1cb13b4d4bb64196fe301c91be'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:25.397593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4668b1cb13b4d4bb64196fe301c91be'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:25.410829 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4668b1cb13b4d4bb64196fe301c91be'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:25.418554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4668b1cb13b4d4bb64196fe301c91be'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:25.419260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4668b1cb13b4d4bb64196fe301c91be'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:25.434306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1081/2156] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:25 2022] DELETE /identity/v3/users/c4668b1cb13b4d4bb64196fe301c91be => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:25.439309 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936ae09-0801-46b8-96bf-1307ecf2434f None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:25.439823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936ae09-0801-46b8-96bf-1307ecf2434f None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:25.439924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936ae09-0801-46b8-96bf-1307ecf2434f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:25.459926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b936ae09-0801-46b8-96bf-1307ecf2434f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:25.461140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b936ae09-0801-46b8-96bf-1307ecf2434f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:25.521292 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1076/2157] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:25.531868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:25.533322 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:25.598305 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669de2b6-6350-4423-8d25-41436b4c4b30 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:25.602423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961631ca-fca9-418a-8e3e-225886804ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:25.603112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961631ca-fca9-418a-8e3e-225886804ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:25.603447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961631ca-fca9-418a-8e3e-225886804ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:25.603761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961631ca-fca9-418a-8e3e-225886804ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:25.614609 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961631ca-fca9-418a-8e3e-225886804ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:25.616296 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961631ca-fca9-418a-8e3e-225886804ed3 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:25.629187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1082/2158] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:25.928047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936ae09-0801-46b8-96bf-1307ecf2434f None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:25.929624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b936ae09-0801-46b8-96bf-1307ecf2434f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:25.948864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936ae09-0801-46b8-96bf-1307ecf2434f None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:25.953975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:25.954619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:25.954894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:25.955196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] PATH_INFO: `/v3/projects/3fd281aef23d49dab9f190c59726d1cd` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:25.956568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd281aef23d49dab9f190c59726d1cd)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:25.958391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:25.960971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:26.007500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd281aef23d49dab9f190c59726d1cd', 'request_id': 'req-751a210d-5b4d-47d6-9549-7b144ca85de6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.015565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd281aef23d49dab9f190c59726d1cd', 'request_id': 'req-751a210d-5b4d-47d6-9549-7b144ca85de6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.024506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd281aef23d49dab9f190c59726d1cd', 'request_id': 'req-751a210d-5b4d-47d6-9549-7b144ca85de6'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.036806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] The token cache is being invalidate because project 3fd281aef23d49dab9f190c59726d1cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:26.037088 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.037800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.039477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1077/2159] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:25 2022] DELETE /identity/v3/projects/3fd281aef23d49dab9f190c59726d1cd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:26.043782 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961631ca-fca9-418a-8e3e-225886804ed3 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:26.055084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961631ca-fca9-418a-8e3e-225886804ed3 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:26.113095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:26.113729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:26.114033 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:26.114302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] PATH_INFO: `/v3/users/6ecddcd886f04843a8344f1c8bf5eaf5` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:26.115463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ecddcd886f04843a8344f1c8bf5eaf5)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:26.117382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:26.130051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:26.176673 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ecddcd886f04843a8344f1c8bf5eaf5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.177424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ecddcd886f04843a8344f1c8bf5eaf5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.191921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ecddcd886f04843a8344f1c8bf5eaf5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.199676 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ecddcd886f04843a8344f1c8bf5eaf5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.200391 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6155c27f-282a-4c5c-8181-50966eb9531b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ecddcd886f04843a8344f1c8bf5eaf5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.214911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1083/2160] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:26 2022] DELETE /identity/v3/users/6ecddcd886f04843a8344f1c8bf5eaf5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:26.535533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:26.537649 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:26.556415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a210d-5b4d-47d6-9549-7b144ca85de6 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:26.629365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:26.630000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:26.630279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:26.630553 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] PATH_INFO: `/v3/projects/ecadeeb6649545e5928fef10e206d959` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:26.632314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecadeeb6649545e5928fef10e206d959)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:26.634492 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:26.699268 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecadeeb6649545e5928fef10e206d959', 'request_id': 'req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.707838 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecadeeb6649545e5928fef10e206d959', 'request_id': 'req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.716435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecadeeb6649545e5928fef10e206d959', 'request_id': 'req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.732798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] The token cache is being invalidate because project ecadeeb6649545e5928fef10e206d959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:26.733209 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.733967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b4f4fd1-dafb-4497-8896-87c4ef7b46d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:26.736647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1078/2161] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:26 2022] DELETE /identity/v3/projects/ecadeeb6649545e5928fef10e206d959 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:12:26.742464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e56e8-07f8-43e2-ab1f-1e89a6605817 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:26.742823 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e56e8-07f8-43e2-ab1f-1e89a6605817 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:26.743189 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e56e8-07f8-43e2-ab1f-1e89a6605817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:26.772346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-9c5e56e8-07f8-43e2-ab1f-1e89a6605817 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:26.773600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-9c5e56e8-07f8-43e2-ab1f-1e89a6605817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:26.872980 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1084/2162] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:26.884212 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ec040-9f09-4553-9f40-2db36996f340 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:26.884645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ec040-9f09-4553-9f40-2db36996f340 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:26.885123 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ec040-9f09-4553-9f40-2db36996f340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:26.910623 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-7f1ec040-9f09-4553-9f40-2db36996f340 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:26.912050 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-7f1ec040-9f09-4553-9f40-2db36996f340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:26.922273 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1079/2163] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:26.926851 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e56e8-07f8-43e2-ab1f-1e89a6605817 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:26.938170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e56e8-07f8-43e2-ab1f-1e89a6605817 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:26.942967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce365-3187-42f4-a25d-3e2ea364f458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:26.943601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce365-3187-42f4-a25d-3e2ea364f458 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:26.943875 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce365-3187-42f4-a25d-3e2ea364f458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:26.944165 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce365-3187-42f4-a25d-3e2ea364f458 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:26.944788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ce365-3187-42f4-a25d-3e2ea364f458 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:12:26.945089 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ce365-3187-42f4-a25d-3e2ea364f458 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:26.946685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ce365-3187-42f4-a25d-3e2ea364f458 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:26.959440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1085/2164] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:12:26.965363 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ec040-9f09-4553-9f40-2db36996f340 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:26.976692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ec040-9f09-4553-9f40-2db36996f340 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:26.982224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44708b-3084-4277-a352-783c84fd606a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:26.982826 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44708b-3084-4277-a352-783c84fd606a admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:26.983463 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44708b-3084-4277-a352-783c84fd606a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:26.983610 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44708b-3084-4277-a352-783c84fd606a admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:26.984354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44708b-3084-4277-a352-783c84fd606a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:26.986662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44708b-3084-4277-a352-783c84fd606a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.003997 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:27.026515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1080/2165] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:26 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:27.031131 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce365-3187-42f4-a25d-3e2ea364f458 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.042800 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce365-3187-42f4-a25d-3e2ea364f458 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.109309 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e32f59-cfd6-479c-a29a-468f190d716e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.110098 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e32f59-cfd6-479c-a29a-468f190d716e admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.110203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e32f59-cfd6-479c-a29a-468f190d716e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.110511 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e32f59-cfd6-479c-a29a-468f190d716e admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.111314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e32f59-cfd6-479c-a29a-468f190d716e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.113611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e32f59-cfd6-479c-a29a-468f190d716e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.144513 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1086/2166] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:27 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:27.148912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44708b-3084-4277-a352-783c84fd606a admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.159581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44708b-3084-4277-a352-783c84fd606a admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.164279 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adb10d-fa91-4f13-9d83-9ca062ec2889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.164872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adb10d-fa91-4f13-9d83-9ca062ec2889 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.165168 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adb10d-fa91-4f13-9d83-9ca062ec2889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.165418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adb10d-fa91-4f13-9d83-9ca062ec2889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.166085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adb10d-fa91-4f13-9d83-9ca062ec2889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.167739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adb10d-fa91-4f13-9d83-9ca062ec2889 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.178445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1081/2167] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:12:27.182690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e32f59-cfd6-479c-a29a-468f190d716e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.189645 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:27.196080 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e32f59-cfd6-479c-a29a-468f190d716e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.206225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281a788-8637-4096-a593-5af5079b81c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.207210 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281a788-8637-4096-a593-5af5079b81c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.207651 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281a788-8637-4096-a593-5af5079b81c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.208306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281a788-8637-4096-a593-5af5079b81c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.209179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281a788-8637-4096-a593-5af5079b81c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.211253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281a788-8637-4096-a593-5af5079b81c6 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.223292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1087/2168] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:27.227622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adb10d-fa91-4f13-9d83-9ca062ec2889 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.240306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adb10d-fa91-4f13-9d83-9ca062ec2889 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.246516 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a18a57-8889-4363-9365-f94406cd670c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.247630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a18a57-8889-4363-9365-f94406cd670c admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.247927 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a18a57-8889-4363-9365-f94406cd670c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.248306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a18a57-8889-4363-9365-f94406cd670c admin admin] PATH_INFO: `/v3/projects/9d8faefdb5c64f31a1de4416dccc7bb1/users/5811bb9527f147e7a7557a7a60645107/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.267004 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a18a57-8889-4363-9365-f94406cd670c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8faefdb5c64f31a1de4416dccc7bb1, user_id=5811bb9527f147e7a7557a7a60645107, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.269756 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a18a57-8889-4363-9365-f94406cd670c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.283784 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1082/2169] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:27 2022] PUT /identity/v3/projects/9d8faefdb5c64f31a1de4416dccc7bb1/users/5811bb9527f147e7a7557a7a60645107/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:27.295613 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a30456-bc84-4ce9-9aa5-3f7371c6477b None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.295894 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a30456-bc84-4ce9-9aa5-3f7371c6477b None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.296167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a30456-bc84-4ce9-9aa5-3f7371c6477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.325406 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e6a30456-bc84-4ce9-9aa5-3f7371c6477b None None] MFA Rules not processed for user `5811bb9527f147e7a7557a7a60645107`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:27.326608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e6a30456-bc84-4ce9-9aa5-3f7371c6477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:27.395206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1088/2170] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:27.399798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a18a57-8889-4363-9365-f94406cd670c admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.461891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a18a57-8889-4363-9365-f94406cd670c admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.466672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31208e-1dd3-4e83-b6d1-72f8331970b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.467316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31208e-1dd3-4e83-b6d1-72f8331970b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.467591 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31208e-1dd3-4e83-b6d1-72f8331970b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.467867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31208e-1dd3-4e83-b6d1-72f8331970b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.468503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31208e-1dd3-4e83-b6d1-72f8331970b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.470456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31208e-1dd3-4e83-b6d1-72f8331970b1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.499156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1083/2171] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:27 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:12:27.503674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a30456-bc84-4ce9-9aa5-3f7371c6477b None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.516692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a30456-bc84-4ce9-9aa5-3f7371c6477b None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.580680 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d632d-dc58-4763-beeb-30f2f29f3073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.581356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d632d-dc58-4763-beeb-30f2f29f3073 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.581665 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d632d-dc58-4763-beeb-30f2f29f3073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.581986 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d632d-dc58-4763-beeb-30f2f29f3073 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.582677 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d632d-dc58-4763-beeb-30f2f29f3073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.584992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d632d-dc58-4763-beeb-30f2f29f3073 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.611250 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1089/2172] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:27 2022] POST /identity/v3/users => generated 303 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:27.615354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31208e-1dd3-4e83-b6d1-72f8331970b1 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.627222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31208e-1dd3-4e83-b6d1-72f8331970b1 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.639745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895c259-d1dc-45b2-99c4-c4c28de1f273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.640251 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895c259-d1dc-45b2-99c4-c4c28de1f273 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.640509 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895c259-d1dc-45b2-99c4-c4c28de1f273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.640825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895c259-d1dc-45b2-99c4-c4c28de1f273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.641476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895c259-d1dc-45b2-99c4-c4c28de1f273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.643195 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895c259-d1dc-45b2-99c4-c4c28de1f273 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.654543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1084/2173] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:27.659432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d632d-dc58-4763-beeb-30f2f29f3073 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.670146 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d632d-dc58-4763-beeb-30f2f29f3073 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.675924 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdcb6b-dccc-420b-8697-2da51492573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.676597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdcb6b-dccc-420b-8697-2da51492573e admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.676878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdcb6b-dccc-420b-8697-2da51492573e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.677150 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdcb6b-dccc-420b-8697-2da51492573e admin admin] PATH_INFO: `/v3/projects/37a676c722054e7e921c9ccc89f144ba/users/aa368c82b9a74c239173ba89086395a4/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.693833 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdcb6b-dccc-420b-8697-2da51492573e admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a676c722054e7e921c9ccc89f144ba, user_id=aa368c82b9a74c239173ba89086395a4, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.696567 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdcb6b-dccc-420b-8697-2da51492573e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.714781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1090/2174] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:27 2022] PUT /identity/v3/projects/37a676c722054e7e921c9ccc89f144ba/users/aa368c82b9a74c239173ba89086395a4/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:27.719770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895c259-d1dc-45b2-99c4-c4c28de1f273 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.731074 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895c259-d1dc-45b2-99c4-c4c28de1f273 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.798111 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98485a5-d9c6-414b-9501-b1c79502036b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.798745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98485a5-d9c6-414b-9501-b1c79502036b admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.799055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98485a5-d9c6-414b-9501-b1c79502036b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.799329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98485a5-d9c6-414b-9501-b1c79502036b admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.800001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98485a5-d9c6-414b-9501-b1c79502036b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.801643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98485a5-d9c6-414b-9501-b1c79502036b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.812637 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1085/2175] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:27.818115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdcb6b-dccc-420b-8697-2da51492573e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:27.830948 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdcb6b-dccc-420b-8697-2da51492573e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:27.836674 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cc9ed-66e2-4e09-b083-b37f3c3389a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:27.837431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cc9ed-66e2-4e09-b083-b37f3c3389a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.837757 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cc9ed-66e2-4e09-b083-b37f3c3389a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.838068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cc9ed-66e2-4e09-b083-b37f3c3389a5 admin admin] PATH_INFO: `/v3/projects/37a676c722054e7e921c9ccc89f144ba/users/aa368c82b9a74c239173ba89086395a4/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.840312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7cc9ed-66e2-4e09-b083-b37f3c3389a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a676c722054e7e921c9ccc89f144ba, user_id=aa368c82b9a74c239173ba89086395a4, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:27.844227 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7cc9ed-66e2-4e09-b083-b37f3c3389a5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:27.857272 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1091/2176] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:27 2022] PUT /identity/v3/projects/37a676c722054e7e921c9ccc89f144ba/users/aa368c82b9a74c239173ba89086395a4/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:27.869744 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d426d7c-931e-416e-8f77-c6ad741f5f39 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:27.870296 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d426d7c-931e-416e-8f77-c6ad741f5f39 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:27.870558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d426d7c-931e-416e-8f77-c6ad741f5f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:27.914558 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-0d426d7c-931e-416e-8f77-c6ad741f5f39 None None] MFA Rules not processed for user `aa368c82b9a74c239173ba89086395a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:27.915970 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0d426d7c-931e-416e-8f77-c6ad741f5f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:27.937598 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:27.996860 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1086/2177] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:28.011800 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cc9ed-66e2-4e09-b083-b37f3c3389a5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:28.014047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1a7cc9ed-66e2-4e09-b083-b37f3c3389a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:28.076922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:28.101558 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cc9ed-66e2-4e09-b083-b37f3c3389a5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:28.105864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b5f18-bfda-465d-84c2-057237ced368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:28.106540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b5f18-bfda-465d-84c2-057237ced368 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:28.106792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b5f18-bfda-465d-84c2-057237ced368 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:28.107300 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b5f18-bfda-465d-84c2-057237ced368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:28.117997 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b5f18-bfda-465d-84c2-057237ced368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:28.119494 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b5f18-bfda-465d-84c2-057237ced368 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:28.130061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1092/2178] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:28 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:28.181499 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d426d7c-931e-416e-8f77-c6ad741f5f39 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:28.192955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d426d7c-931e-416e-8f77-c6ad741f5f39 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:28.196928 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c39578-aa99-46b8-b6ac-d1bc31100a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:28.197540 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c39578-aa99-46b8-b6ac-d1bc31100a83 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:28.197821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c39578-aa99-46b8-b6ac-d1bc31100a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:28.198104 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c39578-aa99-46b8-b6ac-d1bc31100a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:28.291476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c39578-aa99-46b8-b6ac-d1bc31100a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:28.292995 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c39578-aa99-46b8-b6ac-d1bc31100a83 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:28.305451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1087/2179] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:31.726419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b5f18-bfda-465d-84c2-057237ced368 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:31.791903 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b5f18-bfda-465d-84c2-057237ced368 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:31.797269 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:31.797806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:31.798084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:31.798350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] PATH_INFO: `/v3/users/5811bb9527f147e7a7557a7a60645107` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:31.799505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5811bb9527f147e7a7557a7a60645107)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:31.801422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:31.849763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5811bb9527f147e7a7557a7a60645107'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:31.850696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5811bb9527f147e7a7557a7a60645107'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:31.867095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5811bb9527f147e7a7557a7a60645107'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:31.875815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5811bb9527f147e7a7557a7a60645107'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:31.876758 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5811bb9527f147e7a7557a7a60645107'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:31.893059 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1093/2180] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:31 2022] DELETE /identity/v3/users/5811bb9527f147e7a7557a7a60645107 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:31.898522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3dfe3-60aa-4047-a2da-71524771d1a6 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:31.899027 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3dfe3-60aa-4047-a2da-71524771d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:31.899244 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3dfe3-60aa-4047-a2da-71524771d1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:31.922618 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-47d3dfe3-60aa-4047-a2da-71524771d1a6 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:31.924012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-47d3dfe3-60aa-4047-a2da-71524771d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:31.990041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1088/2181] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:32.001916 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:32.004087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:32.068443 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94015620-ac95-4e66-8edd-77ff68e40966 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:32.075975 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677aed94-356a-47b3-a807-819b7f09d5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:32.076618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aed94-356a-47b3-a807-819b7f09d5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:32.076900 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aed94-356a-47b3-a807-819b7f09d5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:32.077171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aed94-356a-47b3-a807-819b7f09d5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:32.087171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677aed94-356a-47b3-a807-819b7f09d5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:32.088643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677aed94-356a-47b3-a807-819b7f09d5b6 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:32.095439 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:32.101029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1094/2182] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:32.425651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3dfe3-60aa-4047-a2da-71524771d1a6 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:32.427174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-47d3dfe3-60aa-4047-a2da-71524771d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:32.441614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3dfe3-60aa-4047-a2da-71524771d1a6 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:32.446950 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:32.447612 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:32.448465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:32.448465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] PATH_INFO: `/v3/projects/9d8faefdb5c64f31a1de4416dccc7bb1` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:32.449325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d8faefdb5c64f31a1de4416dccc7bb1)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:32.451528 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:32.498785 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:32.500327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d8faefdb5c64f31a1de4416dccc7bb1', 'request_id': 'req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.509578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d8faefdb5c64f31a1de4416dccc7bb1', 'request_id': 'req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.518998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d8faefdb5c64f31a1de4416dccc7bb1', 'request_id': 'req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.530775 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] The token cache is being invalidate because project 9d8faefdb5c64f31a1de4416dccc7bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:32.531659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.532418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.534065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1089/2183] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:32 2022] DELETE /identity/v3/projects/9d8faefdb5c64f31a1de4416dccc7bb1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:32.538465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677aed94-356a-47b3-a807-819b7f09d5b6 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:32.548488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677aed94-356a-47b3-a807-819b7f09d5b6 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:32.617053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:32.617803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:32.618137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:32.618356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] PATH_INFO: `/v3/users/aa368c82b9a74c239173ba89086395a4` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:32.619528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa368c82b9a74c239173ba89086395a4)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:32.621480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:32.673304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa368c82b9a74c239173ba89086395a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.674061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa368c82b9a74c239173ba89086395a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.688282 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa368c82b9a74c239173ba89086395a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.697091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa368c82b9a74c239173ba89086395a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.698068 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-40f0acd1-be98-4f0e-bac9-7dae45b1d720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa368c82b9a74c239173ba89086395a4'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:32.713423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1095/2184] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:32 2022] DELETE /identity/v3/users/aa368c82b9a74c239173ba89086395a4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:12:33.011923 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:33.014002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:33.029316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7eafb-04ac-41d4-b61d-befb632ac7b7 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:33.098345 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:33.098998 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.099282 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.099636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] PATH_INFO: `/v3/projects/37a676c722054e7e921c9ccc89f144ba` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.100665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a676c722054e7e921c9ccc89f144ba)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:33.102568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:33.157503 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a676c722054e7e921c9ccc89f144ba', 'request_id': 'req-e3602194-6c68-4d27-9f01-23e54c2366c8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:33.163823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a676c722054e7e921c9ccc89f144ba', 'request_id': 'req-e3602194-6c68-4d27-9f01-23e54c2366c8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:33.172354 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a676c722054e7e921c9ccc89f144ba', 'request_id': 'req-e3602194-6c68-4d27-9f01-23e54c2366c8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:33.183512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] The token cache is being invalidate because project 37a676c722054e7e921c9ccc89f144ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:33.183850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:33.184578 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e3602194-6c68-4d27-9f01-23e54c2366c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:33.186282 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1090/2185] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:33 2022] DELETE /identity/v3/projects/37a676c722054e7e921c9ccc89f144ba => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:33.191679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614dbf3-c779-4eb1-a5da-0918d093994e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.192279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614dbf3-c779-4eb1-a5da-0918d093994e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.192389 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614dbf3-c779-4eb1-a5da-0918d093994e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.216361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-d614dbf3-c779-4eb1-a5da-0918d093994e None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:33.217581 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d614dbf3-c779-4eb1-a5da-0918d093994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:33.289258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1096/2186] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:33.299687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0a5e-4437-4e67-8c43-b2963b9e0f91 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.299973 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0a5e-4437-4e67-8c43-b2963b9e0f91 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.300247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0a5e-4437-4e67-8c43-b2963b9e0f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.321723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-249f0a5e-4437-4e67-8c43-b2963b9e0f91 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:33.323045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-249f0a5e-4437-4e67-8c43-b2963b9e0f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:33.331314 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1091/2187] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:33.336268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614dbf3-c779-4eb1-a5da-0918d093994e None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:33.347628 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614dbf3-c779-4eb1-a5da-0918d093994e None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:33.352905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc1f06-cfe3-4697-a19a-04d08cd10c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:33.353580 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc1f06-cfe3-4697-a19a-04d08cd10c7c admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.353890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc1f06-cfe3-4697-a19a-04d08cd10c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.354210 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc1f06-cfe3-4697-a19a-04d08cd10c7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.354850 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc1f06-cfe3-4697-a19a-04d08cd10c7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:12:33.355242 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc1f06-cfe3-4697-a19a-04d08cd10c7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:33.357113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc1f06-cfe3-4697-a19a-04d08cd10c7c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:33.372285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1097/2188] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:12:33.377207 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f0a5e-4437-4e67-8c43-b2963b9e0f91 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:33.387674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f0a5e-4437-4e67-8c43-b2963b9e0f91 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:33.393635 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33471c-1845-4885-8c85-5ccf3f2e8b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:33.394407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33471c-1845-4885-8c85-5ccf3f2e8b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.394732 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33471c-1845-4885-8c85-5ccf3f2e8b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.395036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33471c-1845-4885-8c85-5ccf3f2e8b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.395819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33471c-1845-4885-8c85-5ccf3f2e8b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:33.397915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33471c-1845-4885-8c85-5ccf3f2e8b05 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:33.428056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1092/2189] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:33 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:33.432034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc1f06-cfe3-4697-a19a-04d08cd10c7c admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:33.443423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc1f06-cfe3-4697-a19a-04d08cd10c7c admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:33.447357 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:33.519808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7af161-f01a-4be2-a04e-5c832b3118d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:33.520584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7af161-f01a-4be2-a04e-5c832b3118d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.520962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7af161-f01a-4be2-a04e-5c832b3118d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.521316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7af161-f01a-4be2-a04e-5c832b3118d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.522107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7af161-f01a-4be2-a04e-5c832b3118d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:33.524278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7af161-f01a-4be2-a04e-5c832b3118d5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:33.549820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1098/2190] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:33 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:33.554124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33471c-1845-4885-8c85-5ccf3f2e8b05 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:33.570464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33471c-1845-4885-8c85-5ccf3f2e8b05 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:33.577862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4e73e-0611-42b7-87d9-4cec1d878ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:33.578804 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4e73e-0611-42b7-87d9-4cec1d878ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.579331 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4e73e-0611-42b7-87d9-4cec1d878ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.579726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4e73e-0611-42b7-87d9-4cec1d878ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.580788 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4e73e-0611-42b7-87d9-4cec1d878ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:33.583609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4e73e-0611-42b7-87d9-4cec1d878ca5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:33.598916 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1093/2191] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:33.603512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7af161-f01a-4be2-a04e-5c832b3118d5 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:33.616043 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7af161-f01a-4be2-a04e-5c832b3118d5 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:33.621318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ed40b-7a92-44b2-b2f9-fde792f553a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:33.621966 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ed40b-7a92-44b2-b2f9-fde792f553a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.622263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ed40b-7a92-44b2-b2f9-fde792f553a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.622568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ed40b-7a92-44b2-b2f9-fde792f553a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.623272 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ed40b-7a92-44b2-b2f9-fde792f553a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:33.625024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ed40b-7a92-44b2-b2f9-fde792f553a9 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:33.638739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1099/2192] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:33.643842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4e73e-0611-42b7-87d9-4cec1d878ca5 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:33.659181 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4e73e-0611-42b7-87d9-4cec1d878ca5 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:33.665003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03603755-f1e5-4e5f-b189-01993bfcdd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:33.665717 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03603755-f1e5-4e5f-b189-01993bfcdd45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.666031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03603755-f1e5-4e5f-b189-01993bfcdd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.666305 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03603755-f1e5-4e5f-b189-01993bfcdd45 admin admin] PATH_INFO: `/v3/projects/afc9b6eb39fe4e579bc43417ab3f537c/users/7585d239a7d242f0b0be89c3f1bbda3f/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.684719 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03603755-f1e5-4e5f-b189-01993bfcdd45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc9b6eb39fe4e579bc43417ab3f537c, user_id=7585d239a7d242f0b0be89c3f1bbda3f, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:33.688308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03603755-f1e5-4e5f-b189-01993bfcdd45 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:33.704067 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1094/2193] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:33 2022] PUT /identity/v3/projects/afc9b6eb39fe4e579bc43417ab3f537c/users/7585d239a7d242f0b0be89c3f1bbda3f/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:33.714499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe3efd-d301-4e86-bb5b-697e0dd2ec30 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.714781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe3efd-d301-4e86-bb5b-697e0dd2ec30 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.715147 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe3efd-d301-4e86-bb5b-697e0dd2ec30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.744930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-1dbe3efd-d301-4e86-bb5b-697e0dd2ec30 None None] MFA Rules not processed for user `7585d239a7d242f0b0be89c3f1bbda3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:33.746153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1dbe3efd-d301-4e86-bb5b-697e0dd2ec30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:33.818798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1100/2194] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:33.823898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03603755-f1e5-4e5f-b189-01993bfcdd45 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:33.829112 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:33.897158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03603755-f1e5-4e5f-b189-01993bfcdd45 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:33.901864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02141c55-ed6e-45d3-9f56-c8183ea11b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:33.902551 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02141c55-ed6e-45d3-9f56-c8183ea11b50 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:33.902834 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02141c55-ed6e-45d3-9f56-c8183ea11b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:33.903163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02141c55-ed6e-45d3-9f56-c8183ea11b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:33.903884 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02141c55-ed6e-45d3-9f56-c8183ea11b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:33.906390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02141c55-ed6e-45d3-9f56-c8183ea11b50 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:33.942002 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1095/2195] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:33 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:33.942864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe3efd-d301-4e86-bb5b-697e0dd2ec30 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:33.954150 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe3efd-d301-4e86-bb5b-697e0dd2ec30 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:34.012376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4de55-23c0-4ee0-a139-f198aee4bc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:34.013100 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4de55-23c0-4ee0-a139-f198aee4bc8e admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:34.013382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4de55-23c0-4ee0-a139-f198aee4bc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:34.013656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4de55-23c0-4ee0-a139-f198aee4bc8e admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:34.014301 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4de55-23c0-4ee0-a139-f198aee4bc8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:34.016825 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4de55-23c0-4ee0-a139-f198aee4bc8e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:34.042615 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1101/2196] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:33 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:34.043990 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02141c55-ed6e-45d3-9f56-c8183ea11b50 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:34.055450 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02141c55-ed6e-45d3-9f56-c8183ea11b50 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:34.060867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65057cd6-ed93-4003-a121-eb8335e5c121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:34.061490 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65057cd6-ed93-4003-a121-eb8335e5c121 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:34.061763 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65057cd6-ed93-4003-a121-eb8335e5c121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:34.062091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65057cd6-ed93-4003-a121-eb8335e5c121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:34.062791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65057cd6-ed93-4003-a121-eb8335e5c121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:34.064656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65057cd6-ed93-4003-a121-eb8335e5c121 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:34.076474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1096/2197] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:12:34.081690 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4de55-23c0-4ee0-a139-f198aee4bc8e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:34.090551 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:34.097464 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4de55-23c0-4ee0-a139-f198aee4bc8e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:34.103490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c16aa4-ec51-43d4-ac58-31675aafa272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:34.104203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c16aa4-ec51-43d4-ac58-31675aafa272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:34.104535 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c16aa4-ec51-43d4-ac58-31675aafa272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:34.104857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c16aa4-ec51-43d4-ac58-31675aafa272 admin admin] PATH_INFO: `/v3/projects/33bf04d35d504135adcb9052b8aa7989/users/ef56b9c3a579491db253fa90eef20840/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:34.122268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c16aa4-ec51-43d4-ac58-31675aafa272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33bf04d35d504135adcb9052b8aa7989, user_id=ef56b9c3a579491db253fa90eef20840, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:34.125325 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c16aa4-ec51-43d4-ac58-31675aafa272 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:34.142067 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1102/2198] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:34 2022] PUT /identity/v3/projects/33bf04d35d504135adcb9052b8aa7989/users/ef56b9c3a579491db253fa90eef20840/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:34.146388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65057cd6-ed93-4003-a121-eb8335e5c121 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:34.157451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65057cd6-ed93-4003-a121-eb8335e5c121 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:34.217802 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9016ce-6a25-46ad-b900-3c6ff2146d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:34.218325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9016ce-6a25-46ad-b900-3c6ff2146d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:34.219196 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9016ce-6a25-46ad-b900-3c6ff2146d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:34.219464 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9016ce-6a25-46ad-b900-3c6ff2146d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:34.220113 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9016ce-6a25-46ad-b900-3c6ff2146d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:34.221751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9016ce-6a25-46ad-b900-3c6ff2146d1b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:34.233322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1097/2199] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:34.237689 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c16aa4-ec51-43d4-ac58-31675aafa272 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:34.249647 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c16aa4-ec51-43d4-ac58-31675aafa272 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:34.254607 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe674ef-6168-416a-8cee-ea909154c04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:34.255316 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe674ef-6168-416a-8cee-ea909154c04c admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:34.255595 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe674ef-6168-416a-8cee-ea909154c04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:34.255868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe674ef-6168-416a-8cee-ea909154c04c admin admin] PATH_INFO: `/v3/projects/33bf04d35d504135adcb9052b8aa7989/users/ef56b9c3a579491db253fa90eef20840/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:34.257808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe674ef-6168-416a-8cee-ea909154c04c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33bf04d35d504135adcb9052b8aa7989, user_id=ef56b9c3a579491db253fa90eef20840, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:34.260791 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe674ef-6168-416a-8cee-ea909154c04c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:34.272167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1103/2200] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:34 2022] PUT /identity/v3/projects/33bf04d35d504135adcb9052b8aa7989/users/ef56b9c3a579491db253fa90eef20840/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:34.283940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a8242-99c1-40b2-84a1-c7656cbc5112 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:34.284222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a8242-99c1-40b2-84a1-c7656cbc5112 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:34.284495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a8242-99c1-40b2-84a1-c7656cbc5112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:34.315056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-227a8242-99c1-40b2-84a1-c7656cbc5112 None None] MFA Rules not processed for user `ef56b9c3a579491db253fa90eef20840`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:34.316286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-227a8242-99c1-40b2-84a1-c7656cbc5112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:34.389013 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:34.399161 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1098/2201] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:34.411091 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe674ef-6168-416a-8cee-ea909154c04c admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:34.413602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4fe674ef-6168-416a-8cee-ea909154c04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:34.480499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe674ef-6168-416a-8cee-ea909154c04c admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:34.484322 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46de67b-c10f-4cc1-bbfe-06c249838877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:34.484931 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46de67b-c10f-4cc1-bbfe-06c249838877 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:34.485211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46de67b-c10f-4cc1-bbfe-06c249838877 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:34.485495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46de67b-c10f-4cc1-bbfe-06c249838877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:34.494667 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46de67b-c10f-4cc1-bbfe-06c249838877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:34.496135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46de67b-c10f-4cc1-bbfe-06c249838877 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:34.506291 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1104/2202] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:34.616775 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a8242-99c1-40b2-84a1-c7656cbc5112 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:34.628597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a8242-99c1-40b2-84a1-c7656cbc5112 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:34.632638 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701bf0b-1740-4ddf-a48b-0ff7b83b2ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:34.633622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701bf0b-1740-4ddf-a48b-0ff7b83b2ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:34.633951 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701bf0b-1740-4ddf-a48b-0ff7b83b2ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:34.634389 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701bf0b-1740-4ddf-a48b-0ff7b83b2ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:34.705911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701bf0b-1740-4ddf-a48b-0ff7b83b2ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:34.707445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701bf0b-1740-4ddf-a48b-0ff7b83b2ec8 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:34.717103 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1099/2203] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:37.500908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46de67b-c10f-4cc1-bbfe-06c249838877 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:37.576166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46de67b-c10f-4cc1-bbfe-06c249838877 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:37.585152 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:37.586108 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:37.586485 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:37.586764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] PATH_INFO: `/v3/users/7585d239a7d242f0b0be89c3f1bbda3f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:37.588103 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7585d239a7d242f0b0be89c3f1bbda3f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:37.590529 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:37.662885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7585d239a7d242f0b0be89c3f1bbda3f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:37.663792 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7585d239a7d242f0b0be89c3f1bbda3f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:37.678584 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7585d239a7d242f0b0be89c3f1bbda3f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:37.684007 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:37.686006 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7585d239a7d242f0b0be89c3f1bbda3f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:37.686889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7585d239a7d242f0b0be89c3f1bbda3f'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:37.701575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1105/2204] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:37 2022] DELETE /identity/v3/users/7585d239a7d242f0b0be89c3f1bbda3f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:37.707849 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f74c6-cf14-400d-8e8c-f74c91b40122 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:37.708517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f74c6-cf14-400d-8e8c-f74c91b40122 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:37.708733 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f74c6-cf14-400d-8e8c-f74c91b40122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:37.730189 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-9b0f74c6-cf14-400d-8e8c-f74c91b40122 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:37.731422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9b0f74c6-cf14-400d-8e8c-f74c91b40122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:37.806694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1100/2205] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:37.817443 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:37.819104 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:37.878612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b361a4-8697-46f1-b825-00ee41494005 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:37.882685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231fd8f-9fc4-40a5-bcc4-6c99a2363635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:37.883591 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231fd8f-9fc4-40a5-bcc4-6c99a2363635 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:37.883868 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231fd8f-9fc4-40a5-bcc4-6c99a2363635 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:37.884141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231fd8f-9fc4-40a5-bcc4-6c99a2363635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:37.893852 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231fd8f-9fc4-40a5-bcc4-6c99a2363635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:37.895380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231fd8f-9fc4-40a5-bcc4-6c99a2363635 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:37.905211 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1106/2206] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:38.219298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f74c6-cf14-400d-8e8c-f74c91b40122 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:38.221055 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9b0f74c6-cf14-400d-8e8c-f74c91b40122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:38.234642 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f74c6-cf14-400d-8e8c-f74c91b40122 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:38.240645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:38.242080 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:38.242535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:38.242896 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] PATH_INFO: `/v3/projects/afc9b6eb39fe4e579bc43417ab3f537c` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:38.244955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc9b6eb39fe4e579bc43417ab3f537c)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:38.248909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:38.312395 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc9b6eb39fe4e579bc43417ab3f537c', 'request_id': 'req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.319878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc9b6eb39fe4e579bc43417ab3f537c', 'request_id': 'req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.330085 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc9b6eb39fe4e579bc43417ab3f537c', 'request_id': 'req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.342911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] The token cache is being invalidate because project afc9b6eb39fe4e579bc43417ab3f537c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:38.343754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.344445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.345955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1101/2207] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:38 2022] DELETE /identity/v3/projects/afc9b6eb39fe4e579bc43417ab3f537c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:38.350201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231fd8f-9fc4-40a5-bcc4-6c99a2363635 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:38.361597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231fd8f-9fc4-40a5-bcc4-6c99a2363635 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:38.427120 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:38.427753 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:38.428028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:38.428295 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] PATH_INFO: `/v3/users/ef56b9c3a579491db253fa90eef20840` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:38.429442 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef56b9c3a579491db253fa90eef20840)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:38.431414 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:38.487923 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef56b9c3a579491db253fa90eef20840'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.488731 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef56b9c3a579491db253fa90eef20840'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.504016 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef56b9c3a579491db253fa90eef20840'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.515655 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef56b9c3a579491db253fa90eef20840'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.518346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-c9686657-75db-4166-bbea-80f87606b0af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef56b9c3a579491db253fa90eef20840'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:38.537012 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1107/2208] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:38 2022] DELETE /identity/v3/users/ef56b9c3a579491db253fa90eef20840 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:38.884888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:38.886525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:38.903647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca5c80-9bc8-4fef-9835-a57bfec99cf8 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:38.928727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:38.984390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:38.985124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:38.985466 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:38.985819 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] PATH_INFO: `/v3/projects/33bf04d35d504135adcb9052b8aa7989` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:38.987126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=33bf04d35d504135adcb9052b8aa7989)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:38.989418 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:39.046776 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33bf04d35d504135adcb9052b8aa7989', 'request_id': 'req-6b70b438-9779-4700-b587-d95e7c791d7c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:39.053456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33bf04d35d504135adcb9052b8aa7989', 'request_id': 'req-6b70b438-9779-4700-b587-d95e7c791d7c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:39.061322 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33bf04d35d504135adcb9052b8aa7989', 'request_id': 'req-6b70b438-9779-4700-b587-d95e7c791d7c'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:39.078131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] The token cache is being invalidate because project 33bf04d35d504135adcb9052b8aa7989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:39.078473 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:39.080198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6b70b438-9779-4700-b587-d95e7c791d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:39.081779 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1102/2209] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:38 2022] DELETE /identity/v3/projects/33bf04d35d504135adcb9052b8aa7989 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:12:39.087285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba49089-fd6e-45b7-89f0-69e7cdf9f4b1 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.087576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba49089-fd6e-45b7-89f0-69e7cdf9f4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.087856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba49089-fd6e-45b7-89f0-69e7cdf9f4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.108292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-cba49089-fd6e-45b7-89f0-69e7cdf9f4b1 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:39.109532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cba49089-fd6e-45b7-89f0-69e7cdf9f4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:39.115952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:39.181110 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1108/2210] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:39.190759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e684fb-37eb-4ddc-b442-8b05791815b4 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.191140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e684fb-37eb-4ddc-b442-8b05791815b4 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.191437 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e684fb-37eb-4ddc-b442-8b05791815b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.214297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b6e684fb-37eb-4ddc-b442-8b05791815b4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:39.215655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b6e684fb-37eb-4ddc-b442-8b05791815b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:39.224353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1103/2211] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:39.229845 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba49089-fd6e-45b7-89f0-69e7cdf9f4b1 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:39.242912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba49089-fd6e-45b7-89f0-69e7cdf9f4b1 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:39.248133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745eb82-4ea2-43aa-b5b5-d5b8d9e854a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:39.248806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745eb82-4ea2-43aa-b5b5-d5b8d9e854a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.249108 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745eb82-4ea2-43aa-b5b5-d5b8d9e854a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.250037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745eb82-4ea2-43aa-b5b5-d5b8d9e854a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.250747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745eb82-4ea2-43aa-b5b5-d5b8d9e854a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:12:39.251127 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745eb82-4ea2-43aa-b5b5-d5b8d9e854a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:39.252956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745eb82-4ea2-43aa-b5b5-d5b8d9e854a8 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:39.267039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1109/2212] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:12:39.271803 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e684fb-37eb-4ddc-b442-8b05791815b4 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:39.281574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e684fb-37eb-4ddc-b442-8b05791815b4 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:39.286376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915ead1-fc6a-403d-ae77-a6d579892ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:39.287108 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915ead1-fc6a-403d-ae77-a6d579892ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.287391 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915ead1-fc6a-403d-ae77-a6d579892ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.287663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915ead1-fc6a-403d-ae77-a6d579892ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.288307 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915ead1-fc6a-403d-ae77-a6d579892ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:39.290229 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915ead1-fc6a-403d-ae77-a6d579892ee7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:39.322821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1104/2213] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:39 2022] POST /identity/v3/projects => generated 409 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:12:39.327989 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745eb82-4ea2-43aa-b5b5-d5b8d9e854a8 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:39.341380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745eb82-4ea2-43aa-b5b5-d5b8d9e854a8 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:39.411079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558b8f9-7e79-45e0-a4ac-598225ad43d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:39.411802 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558b8f9-7e79-45e0-a4ac-598225ad43d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.412112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558b8f9-7e79-45e0-a4ac-598225ad43d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.412417 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558b8f9-7e79-45e0-a4ac-598225ad43d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.413157 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558b8f9-7e79-45e0-a4ac-598225ad43d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:39.415425 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558b8f9-7e79-45e0-a4ac-598225ad43d7 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:39.444200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1110/2214] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:39 2022] POST /identity/v3/users => generated 305 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:39.449127 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915ead1-fc6a-403d-ae77-a6d579892ee7 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:39.462517 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915ead1-fc6a-403d-ae77-a6d579892ee7 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:39.475252 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5fe27-a3e6-456a-81ef-8c597c5223af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:39.475968 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5fe27-a3e6-456a-81ef-8c597c5223af admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.476152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5fe27-a3e6-456a-81ef-8c597c5223af admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.476421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5fe27-a3e6-456a-81ef-8c597c5223af admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.477065 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5fe27-a3e6-456a-81ef-8c597c5223af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:39.478901 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5fe27-a3e6-456a-81ef-8c597c5223af admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:39.490052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1105/2215] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:39.495505 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558b8f9-7e79-45e0-a4ac-598225ad43d7 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:39.510334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558b8f9-7e79-45e0-a4ac-598225ad43d7 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:39.517367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559bced-0c25-4181-989a-e8655991a0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:39.518326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559bced-0c25-4181-989a-e8655991a0ff admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.518650 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559bced-0c25-4181-989a-e8655991a0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.519010 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559bced-0c25-4181-989a-e8655991a0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.519784 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559bced-0c25-4181-989a-e8655991a0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:39.521705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559bced-0c25-4181-989a-e8655991a0ff admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:39.534910 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1111/2216] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:39.539479 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5fe27-a3e6-456a-81ef-8c597c5223af admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:39.552922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5fe27-a3e6-456a-81ef-8c597c5223af admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:39.558224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2dcff-f031-464f-9d1c-12dbfb957c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:39.558908 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2dcff-f031-464f-9d1c-12dbfb957c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.559234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2dcff-f031-464f-9d1c-12dbfb957c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.559539 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2dcff-f031-464f-9d1c-12dbfb957c82 admin admin] PATH_INFO: `/v3/projects/08e5f72020f14415af8b2a1f71da4b62/users/194509f15d024ba3bb8c0309259b36f2/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.578890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2dcff-f031-464f-9d1c-12dbfb957c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e5f72020f14415af8b2a1f71da4b62, user_id=194509f15d024ba3bb8c0309259b36f2, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:39.583586 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2dcff-f031-464f-9d1c-12dbfb957c82 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:39.611980 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1106/2217] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:39 2022] PUT /identity/v3/projects/08e5f72020f14415af8b2a1f71da4b62/users/194509f15d024ba3bb8c0309259b36f2/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:39.621992 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d839ce-220d-4a3d-9a96-857ee7835125 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.622278 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d839ce-220d-4a3d-9a96-857ee7835125 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.622554 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d839ce-220d-4a3d-9a96-857ee7835125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.652139 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-d9d839ce-220d-4a3d-9a96-857ee7835125 None None] MFA Rules not processed for user `194509f15d024ba3bb8c0309259b36f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:39.654597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d9d839ce-220d-4a3d-9a96-857ee7835125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:39.729287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1112/2218] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:39 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:39.734443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2dcff-f031-464f-9d1c-12dbfb957c82 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:39.806266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:39.822806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2dcff-f031-464f-9d1c-12dbfb957c82 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:39.829724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ab16f-8c69-4b0b-a0ad-b228f32b4819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:39.830592 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ab16f-8c69-4b0b-a0ad-b228f32b4819 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.831033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ab16f-8c69-4b0b-a0ad-b228f32b4819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.831442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ab16f-8c69-4b0b-a0ad-b228f32b4819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.832384 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ab16f-8c69-4b0b-a0ad-b228f32b4819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:39.835568 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ab16f-8c69-4b0b-a0ad-b228f32b4819 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:39.868594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1107/2219] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:39 2022] POST /identity/v3/projects => generated 407 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:39.872467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d839ce-220d-4a3d-9a96-857ee7835125 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:39.885292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d839ce-220d-4a3d-9a96-857ee7835125 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:39.890045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:39.946539 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecaa76-f910-4b92-b19b-b1ed0424fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:39.946864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecaa76-f910-4b92-b19b-b1ed0424fbc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:39.947172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecaa76-f910-4b92-b19b-b1ed0424fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:39.947478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecaa76-f910-4b92-b19b-b1ed0424fbc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:39.948117 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecaa76-f910-4b92-b19b-b1ed0424fbc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:39.950029 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecaa76-f910-4b92-b19b-b1ed0424fbc0 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:39.978502 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1113/2220] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:39 2022] POST /identity/v3/users => generated 310 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:39.979807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ab16f-8c69-4b0b-a0ad-b228f32b4819 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:39.990237 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ab16f-8c69-4b0b-a0ad-b228f32b4819 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:40.000160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99f782-75f4-4333-8de1-646fa9dc44d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:40.000782 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99f782-75f4-4333-8de1-646fa9dc44d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:40.001059 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99f782-75f4-4333-8de1-646fa9dc44d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:40.001328 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99f782-75f4-4333-8de1-646fa9dc44d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:40.002010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99f782-75f4-4333-8de1-646fa9dc44d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:40.003831 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99f782-75f4-4333-8de1-646fa9dc44d7 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:40.020430 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1108/2221] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:12:40.021603 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecaa76-f910-4b92-b19b-b1ed0424fbc0 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:40.032322 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecaa76-f910-4b92-b19b-b1ed0424fbc0 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:40.037319 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04018cec-baf0-4491-9ba1-a2f9582296b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:40.037982 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04018cec-baf0-4491-9ba1-a2f9582296b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:40.038266 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04018cec-baf0-4491-9ba1-a2f9582296b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:40.038543 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04018cec-baf0-4491-9ba1-a2f9582296b1 admin admin] PATH_INFO: `/v3/projects/659f66a91ea4497bbfaf6e06c1a929a1/users/5bf050ee47aa4dd0adcfcc8f4e195761/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:40.058248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04018cec-baf0-4491-9ba1-a2f9582296b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=659f66a91ea4497bbfaf6e06c1a929a1, user_id=5bf050ee47aa4dd0adcfcc8f4e195761, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:40.060983 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04018cec-baf0-4491-9ba1-a2f9582296b1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:40.076655 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1114/2222] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:40 2022] PUT /identity/v3/projects/659f66a91ea4497bbfaf6e06c1a929a1/users/5bf050ee47aa4dd0adcfcc8f4e195761/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:40.081415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99f782-75f4-4333-8de1-646fa9dc44d7 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:40.092993 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99f782-75f4-4333-8de1-646fa9dc44d7 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:40.159652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2da6e-46c3-47b5-a55e-bda9f122f34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:40.160266 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2da6e-46c3-47b5-a55e-bda9f122f34d admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:40.160547 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2da6e-46c3-47b5-a55e-bda9f122f34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:40.160829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2da6e-46c3-47b5-a55e-bda9f122f34d admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:40.161484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2da6e-46c3-47b5-a55e-bda9f122f34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:40.163151 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2da6e-46c3-47b5-a55e-bda9f122f34d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:40.174233 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1109/2223] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:40.178657 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04018cec-baf0-4491-9ba1-a2f9582296b1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:40.189963 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04018cec-baf0-4491-9ba1-a2f9582296b1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:40.195096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fd8e3-3d33-47ca-a8d2-95af39fa702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:40.195761 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd8e3-3d33-47ca-a8d2-95af39fa702c admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:40.196035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd8e3-3d33-47ca-a8d2-95af39fa702c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:40.196306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd8e3-3d33-47ca-a8d2-95af39fa702c admin admin] PATH_INFO: `/v3/projects/659f66a91ea4497bbfaf6e06c1a929a1/users/5bf050ee47aa4dd0adcfcc8f4e195761/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:40.198250 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fd8e3-3d33-47ca-a8d2-95af39fa702c admin admin] RBAC: Authorizing `identity:create_grant(project_id=659f66a91ea4497bbfaf6e06c1a929a1, user_id=5bf050ee47aa4dd0adcfcc8f4e195761, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:40.200934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fd8e3-3d33-47ca-a8d2-95af39fa702c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:40.216615 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1115/2224] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:40 2022] PUT /identity/v3/projects/659f66a91ea4497bbfaf6e06c1a929a1/users/5bf050ee47aa4dd0adcfcc8f4e195761/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:40.228618 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fbd46-6f0c-4561-87d0-8228bdd03928 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:40.228924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fbd46-6f0c-4561-87d0-8228bdd03928 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:40.229185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fbd46-6f0c-4561-87d0-8228bdd03928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:40.260631 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-710fbd46-6f0c-4561-87d0-8228bdd03928 None None] MFA Rules not processed for user `5bf050ee47aa4dd0adcfcc8f4e195761`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:40.261850 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-710fbd46-6f0c-4561-87d0-8228bdd03928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:40.336541 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1110/2225] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:40 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:40.347873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fd8e3-3d33-47ca-a8d2-95af39fa702c admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:40.350109 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-548fd8e3-3d33-47ca-a8d2-95af39fa702c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:40.422220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fd8e3-3d33-47ca-a8d2-95af39fa702c admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:40.426661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb4ed-7f11-4f20-b9a7-05ec0722cc60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:40.427326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb4ed-7f11-4f20-b9a7-05ec0722cc60 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:40.427600 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb4ed-7f11-4f20-b9a7-05ec0722cc60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:40.427865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb4ed-7f11-4f20-b9a7-05ec0722cc60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:40.437856 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eb4ed-7f11-4f20-b9a7-05ec0722cc60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:40.439313 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eb4ed-7f11-4f20-b9a7-05ec0722cc60 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:40.449396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1116/2226] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:40 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:40.643512 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fbd46-6f0c-4561-87d0-8228bdd03928 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:40.656560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fbd46-6f0c-4561-87d0-8228bdd03928 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:40.664912 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91a2d0-562e-47d5-92f8-1dc576585d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:40.665530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91a2d0-562e-47d5-92f8-1dc576585d6b service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:40.665806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91a2d0-562e-47d5-92f8-1dc576585d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:40.666079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91a2d0-562e-47d5-92f8-1dc576585d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:40.717836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:40.737249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91a2d0-562e-47d5-92f8-1dc576585d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:40.738710 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91a2d0-562e-47d5-92f8-1dc576585d6b service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:40.749081 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1111/2227] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:40 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:12:48.260448 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb4ed-7f11-4f20-b9a7-05ec0722cc60 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:48.339634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb4ed-7f11-4f20-b9a7-05ec0722cc60 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:48.346063 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:48.346716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:48.347031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:48.347310 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] PATH_INFO: `/v3/users/194509f15d024ba3bb8c0309259b36f2` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:48.348740 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=194509f15d024ba3bb8c0309259b36f2)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:48.351114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:48.355123 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:48.417136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '194509f15d024ba3bb8c0309259b36f2'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:48.417955 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '194509f15d024ba3bb8c0309259b36f2'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:48.440144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '194509f15d024ba3bb8c0309259b36f2'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:48.447582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '194509f15d024ba3bb8c0309259b36f2'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:48.448333 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '194509f15d024ba3bb8c0309259b36f2'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:48.463686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1117/2228] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:48 2022] DELETE /identity/v3/users/194509f15d024ba3bb8c0309259b36f2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:48.469033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184d9d-a533-43df-818c-ca77c2fe6790 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:48.469582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184d9d-a533-43df-818c-ca77c2fe6790 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:48.469582 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184d9d-a533-43df-818c-ca77c2fe6790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:48.491402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2a184d9d-a533-43df-818c-ca77c2fe6790 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:48.492621 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2a184d9d-a533-43df-818c-ca77c2fe6790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:48.561801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1112/2229] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:48.575206 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:48.577038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:48.650293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520801e-91c1-4aa8-9740-3fc4afe08a6a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:48.654368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff084f-7c60-4416-89c2-c55e16c2c668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:48.655809 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff084f-7c60-4416-89c2-c55e16c2c668 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:48.656176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff084f-7c60-4416-89c2-c55e16c2c668 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:48.656467 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff084f-7c60-4416-89c2-c55e16c2c668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:48.667276 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff084f-7c60-4416-89c2-c55e16c2c668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:48.668715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff084f-7c60-4416-89c2-c55e16c2c668 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:48.679876 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1118/2230] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:49.019534 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184d9d-a533-43df-818c-ca77c2fe6790 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:49.021074 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2a184d9d-a533-43df-818c-ca77c2fe6790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:49.034875 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184d9d-a533-43df-818c-ca77c2fe6790 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:49.041403 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:49.041799 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:49.042390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:49.042390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] PATH_INFO: `/v3/projects/08e5f72020f14415af8b2a1f71da4b62` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:49.044226 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e5f72020f14415af8b2a1f71da4b62)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:49.047705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:49.112171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e5f72020f14415af8b2a1f71da4b62', 'request_id': 'req-fb1f6c17-2755-4378-a4f0-dd5958b09688'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.123511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e5f72020f14415af8b2a1f71da4b62', 'request_id': 'req-fb1f6c17-2755-4378-a4f0-dd5958b09688'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.132576 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e5f72020f14415af8b2a1f71da4b62', 'request_id': 'req-fb1f6c17-2755-4378-a4f0-dd5958b09688'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.147967 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] The token cache is being invalidate because project 08e5f72020f14415af8b2a1f71da4b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:49.148267 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.148993 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.151392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1113/2231] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:49 2022] DELETE /identity/v3/projects/08e5f72020f14415af8b2a1f71da4b62 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:49.155769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff084f-7c60-4416-89c2-c55e16c2c668 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:49.179026 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff084f-7c60-4416-89c2-c55e16c2c668 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:49.260350 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:49.267298 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:49.268167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:49.268582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] PATH_INFO: `/v3/users/5bf050ee47aa4dd0adcfcc8f4e195761` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:49.270061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf050ee47aa4dd0adcfcc8f4e195761)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:49.272261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:49.330232 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:49.363702 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf050ee47aa4dd0adcfcc8f4e195761'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.364346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf050ee47aa4dd0adcfcc8f4e195761'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.388661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf050ee47aa4dd0adcfcc8f4e195761'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.399061 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf050ee47aa4dd0adcfcc8f4e195761'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.399949 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-797ef127-f072-424f-8aa7-462ee1f577c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf050ee47aa4dd0adcfcc8f4e195761'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:49.417984 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1119/2232] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:49 2022] DELETE /identity/v3/users/5bf050ee47aa4dd0adcfcc8f4e195761 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:49.846222 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:49.848720 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:49.868291 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f6c17-2755-4378-a4f0-dd5958b09688 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:49.941278 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:49.941911 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:49.942185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:49.942449 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] PATH_INFO: `/v3/projects/659f66a91ea4497bbfaf6e06c1a929a1` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:49.943606 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=659f66a91ea4497bbfaf6e06c1a929a1)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:49.945759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:49.952800 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:50.004217 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659f66a91ea4497bbfaf6e06c1a929a1', 'request_id': 'req-7a6d6538-164b-454f-9999-6b49244cf3cc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:50.010919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659f66a91ea4497bbfaf6e06c1a929a1', 'request_id': 'req-7a6d6538-164b-454f-9999-6b49244cf3cc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:50.020504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659f66a91ea4497bbfaf6e06c1a929a1', 'request_id': 'req-7a6d6538-164b-454f-9999-6b49244cf3cc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:50.032971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] The token cache is being invalidate because project 659f66a91ea4497bbfaf6e06c1a929a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:50.033190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:50.033836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7a6d6538-164b-454f-9999-6b49244cf3cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:50.035392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1114/2233] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:49 2022] DELETE /identity/v3/projects/659f66a91ea4497bbfaf6e06c1a929a1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:50.040345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5ca1f-6e1e-4c6f-b931-cd21d194a757 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.040632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5ca1f-6e1e-4c6f-b931-cd21d194a757 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.040949 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5ca1f-6e1e-4c6f-b931-cd21d194a757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.070926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-14f5ca1f-6e1e-4c6f-b931-cd21d194a757 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:50.072765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-14f5ca1f-6e1e-4c6f-b931-cd21d194a757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:50.163739 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1120/2234] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:50.173655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefa93c-3b33-4d89-b15c-16d2d8027ae2 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.174047 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefa93c-3b33-4d89-b15c-16d2d8027ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.174429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefa93c-3b33-4d89-b15c-16d2d8027ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.195668 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-4eefa93c-3b33-4d89-b15c-16d2d8027ae2 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:50.196906 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4eefa93c-3b33-4d89-b15c-16d2d8027ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:50.206903 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1115/2235] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:12:50.210899 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5ca1f-6e1e-4c6f-b931-cd21d194a757 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:50.222433 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5ca1f-6e1e-4c6f-b931-cd21d194a757 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:50.227137 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b871ce-91de-4194-a6e4-be697c8d41f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:50.227775 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b871ce-91de-4194-a6e4-be697c8d41f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.228055 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b871ce-91de-4194-a6e4-be697c8d41f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.228330 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b871ce-91de-4194-a6e4-be697c8d41f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.228981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b871ce-91de-4194-a6e4-be697c8d41f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:12:50.229314 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b871ce-91de-4194-a6e4-be697c8d41f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:50.230921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b871ce-91de-4194-a6e4-be697c8d41f1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:50.242918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1121/2236] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:12:50.248024 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefa93c-3b33-4d89-b15c-16d2d8027ae2 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:50.257628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefa93c-3b33-4d89-b15c-16d2d8027ae2 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:50.262688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a40ac0-d0ab-4e5f-96f3-1f52cdcc366a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:50.263326 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a40ac0-d0ab-4e5f-96f3-1f52cdcc366a admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.263604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a40ac0-d0ab-4e5f-96f3-1f52cdcc366a admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.263872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a40ac0-d0ab-4e5f-96f3-1f52cdcc366a admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.265037 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a40ac0-d0ab-4e5f-96f3-1f52cdcc366a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:50.266944 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a40ac0-d0ab-4e5f-96f3-1f52cdcc366a admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:50.300894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1116/2237] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:50 2022] POST /identity/v3/projects => generated 357 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:50.308781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b871ce-91de-4194-a6e4-be697c8d41f1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:50.319562 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b871ce-91de-4194-a6e4-be697c8d41f1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:50.393035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d74d-8396-431f-9e14-6073ff7acb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:50.393766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d74d-8396-431f-9e14-6073ff7acb03 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.394140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d74d-8396-431f-9e14-6073ff7acb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.394504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d74d-8396-431f-9e14-6073ff7acb03 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.395275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973d74d-8396-431f-9e14-6073ff7acb03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:50.397382 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973d74d-8396-431f-9e14-6073ff7acb03 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:50.425891 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1122/2238] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:50 2022] POST /identity/v3/users => generated 279 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:50.426736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a40ac0-d0ab-4e5f-96f3-1f52cdcc366a admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:50.437242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a40ac0-d0ab-4e5f-96f3-1f52cdcc366a admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:50.443024 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687976e8-6841-437f-8203-bf9c32cf3494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:50.443636 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687976e8-6841-437f-8203-bf9c32cf3494 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.443924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687976e8-6841-437f-8203-bf9c32cf3494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.444188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687976e8-6841-437f-8203-bf9c32cf3494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.445049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687976e8-6841-437f-8203-bf9c32cf3494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:50.446433 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687976e8-6841-437f-8203-bf9c32cf3494 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:50.459031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1117/2239] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:12:50.462219 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d74d-8396-431f-9e14-6073ff7acb03 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:50.474538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d74d-8396-431f-9e14-6073ff7acb03 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:50.480176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc524ad5-528e-42c4-a17f-404de6f5dea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:50.480926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc524ad5-528e-42c4-a17f-404de6f5dea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.481297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc524ad5-528e-42c4-a17f-404de6f5dea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.481664 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc524ad5-528e-42c4-a17f-404de6f5dea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.483490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc524ad5-528e-42c4-a17f-404de6f5dea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:50.486097 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc524ad5-528e-42c4-a17f-404de6f5dea2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:50.500762 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1123/2240] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:50.505038 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687976e8-6841-437f-8203-bf9c32cf3494 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:50.524071 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687976e8-6841-437f-8203-bf9c32cf3494 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:50.528544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d189e-587b-4ac9-8fbd-a77e54a753cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:50.528544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d189e-587b-4ac9-8fbd-a77e54a753cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.528544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d189e-587b-4ac9-8fbd-a77e54a753cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.528544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d189e-587b-4ac9-8fbd-a77e54a753cb admin admin] PATH_INFO: `/v3/projects/ce974235218344deb1c59af207518770/users/6c6b1f4d700a4d8996bae825e3db0637/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.552223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d189e-587b-4ac9-8fbd-a77e54a753cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce974235218344deb1c59af207518770, user_id=6c6b1f4d700a4d8996bae825e3db0637, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:50.555019 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d189e-587b-4ac9-8fbd-a77e54a753cb admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:50.581991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1118/2241] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:50 2022] PUT /identity/v3/projects/ce974235218344deb1c59af207518770/users/6c6b1f4d700a4d8996bae825e3db0637/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:50.589249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc594c-deca-44a1-a5e5-ca5f3819cd56 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.589549 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc594c-deca-44a1-a5e5-ca5f3819cd56 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.589806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc594c-deca-44a1-a5e5-ca5f3819cd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.610626 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:50.621588 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-49bc594c-deca-44a1-a5e5-ca5f3819cd56 None None] MFA Rules not processed for user `6c6b1f4d700a4d8996bae825e3db0637`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:50.623491 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-49bc594c-deca-44a1-a5e5-ca5f3819cd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:50.695921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1124/2242] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:50.733890 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d189e-587b-4ac9-8fbd-a77e54a753cb admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:50.735971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-773d189e-587b-4ac9-8fbd-a77e54a753cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:50.805426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d189e-587b-4ac9-8fbd-a77e54a753cb admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:50.809041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb4163-5585-4f6c-8608-dccfd81294a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:50.809638 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb4163-5585-4f6c-8608-dccfd81294a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.809959 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb4163-5585-4f6c-8608-dccfd81294a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.810346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb4163-5585-4f6c-8608-dccfd81294a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.821688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb4163-5585-4f6c-8608-dccfd81294a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:50.823158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb4163-5585-4f6c-8608-dccfd81294a2 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:50.836158 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1119/2243] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:50.852112 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc594c-deca-44a1-a5e5-ca5f3819cd56 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:50.853603 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-49bc594c-deca-44a1-a5e5-ca5f3819cd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:50.853718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb4163-5585-4f6c-8608-dccfd81294a2 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:50.892415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:50.927924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb4163-5585-4f6c-8608-dccfd81294a2 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:50.933412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:50.934075 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.934476 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.934691 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] PATH_INFO: `/v3/users/6c6b1f4d700a4d8996bae825e3db0637` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.935420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc594c-deca-44a1-a5e5-ca5f3819cd56 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:50.935964 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c6b1f4d700a4d8996bae825e3db0637)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:50.938465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:50.942959 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29782c-491f-4adb-bca1-067ac4dc7bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:50.943858 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29782c-491f-4adb-bca1-067ac4dc7bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:50.944268 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29782c-491f-4adb-bca1-067ac4dc7bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:50.944669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29782c-491f-4adb-bca1-067ac4dc7bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:50.947258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-be29782c-491f-4adb-bca1-067ac4dc7bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:50.966431 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29782c-491f-4adb-bca1-067ac4dc7bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:50.970267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29782c-491f-4adb-bca1-067ac4dc7bf1 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:50.989420 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1125/2244] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:51.011773 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c6b1f4d700a4d8996bae825e3db0637'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.013552 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c6b1f4d700a4d8996bae825e3db0637'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.033414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c6b1f4d700a4d8996bae825e3db0637'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.042875 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c6b1f4d700a4d8996bae825e3db0637'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.053544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c6b1f4d700a4d8996bae825e3db0637'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.063963 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1120/2245] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:50 2022] DELETE /identity/v3/users/6c6b1f4d700a4d8996bae825e3db0637 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:51.070892 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c212-ccd6-4ea1-b765-1fdbca6fa9ed None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:51.072011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c212-ccd6-4ea1-b765-1fdbca6fa9ed None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:51.072425 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c212-ccd6-4ea1-b765-1fdbca6fa9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:51.100437 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-bb41c212-ccd6-4ea1-b765-1fdbca6fa9ed None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:51.101814 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-bb41c212-ccd6-4ea1-b765-1fdbca6fa9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:51.173240 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1126/2246] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:51.190025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:51.195033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:51.259796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fc423-a0a4-407f-a22e-5ba5d9bdc95d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:51.263716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501647e-45a9-4509-b1a8-ef444ba7d33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:51.264361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501647e-45a9-4509-b1a8-ef444ba7d33b service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:51.264656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501647e-45a9-4509-b1a8-ef444ba7d33b service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:51.264910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501647e-45a9-4509-b1a8-ef444ba7d33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:51.274926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501647e-45a9-4509-b1a8-ef444ba7d33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:51.276412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501647e-45a9-4509-b1a8-ef444ba7d33b service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:51.288016 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1121/2247] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 19:12:51.576878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41c212-ccd6-4ea1-b765-1fdbca6fa9ed None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:51.578403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-bb41c212-ccd6-4ea1-b765-1fdbca6fa9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:51.596703 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41c212-ccd6-4ea1-b765-1fdbca6fa9ed None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:51.602058 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:51.602698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:51.602958 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:51.603407 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] PATH_INFO: `/v3/projects/ce974235218344deb1c59af207518770` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:51.604509 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce974235218344deb1c59af207518770)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:51.606469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:51.649361 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:51.661111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce974235218344deb1c59af207518770', 'request_id': 'req-39644c0a-6025-43d1-b2d8-0853dd9416ef'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.672462 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce974235218344deb1c59af207518770', 'request_id': 'req-39644c0a-6025-43d1-b2d8-0853dd9416ef'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.679327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce974235218344deb1c59af207518770', 'request_id': 'req-39644c0a-6025-43d1-b2d8-0853dd9416ef'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.691447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] The token cache is being invalidate because project ce974235218344deb1c59af207518770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:51.691735 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.692392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:51.693890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1127/2248] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:51 2022] DELETE /identity/v3/projects/ce974235218344deb1c59af207518770 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:53.562329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501647e-45a9-4509-b1a8-ef444ba7d33b service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:53.563864 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-4501647e-45a9-4509-b1a8-ef444ba7d33b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:53.578589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501647e-45a9-4509-b1a8-ef444ba7d33b service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:53.635259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b0059-67e7-4f61-af4d-e5cc39648933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:53.635909 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b0059-67e7-4f61-af4d-e5cc39648933 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:53.636194 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b0059-67e7-4f61-af4d-e5cc39648933 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:53.636469 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b0059-67e7-4f61-af4d-e5cc39648933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:53.638310 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-ec8b0059-67e7-4f61-af4d-e5cc39648933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:53.650573 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:53.731751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b0059-67e7-4f61-af4d-e5cc39648933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:53.731751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b0059-67e7-4f61-af4d-e5cc39648933 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:53.743856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1122/2249] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:54.849477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:54.851148 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:54.871335 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39644c0a-6025-43d1-b2d8-0853dd9416ef admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:54.884855 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:54.885777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:54.886053 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:54.886325 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] PATH_INFO: `/v3/users/a25c8936b368477880f429e818a321fa` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:54.887698 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a25c8936b368477880f429e818a321fa)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:54.894718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:54.956403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a25c8936b368477880f429e818a321fa'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:54.957136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a25c8936b368477880f429e818a321fa'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:54.972257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a25c8936b368477880f429e818a321fa'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:54.981532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a25c8936b368477880f429e818a321fa'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:54.982281 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-48e105c3-b7d2-45ac-8750-7c79e75a90b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a25c8936b368477880f429e818a321fa'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:55.005510 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1128/2250] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:54 2022] DELETE /identity/v3/users/a25c8936b368477880f429e818a321fa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:55.211696 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b0059-67e7-4f61-af4d-e5cc39648933 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:55.227489 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b0059-67e7-4f61-af4d-e5cc39648933 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:55.299286 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:55.300043 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:55.300379 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:55.300716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] PATH_INFO: `/v3/projects/526aaf60dd6c42808951c5f48f8fabd2` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:55.302393 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=526aaf60dd6c42808951c5f48f8fabd2)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:55.304497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:55.372026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '526aaf60dd6c42808951c5f48f8fabd2', 'request_id': 'req-6f042893-11b8-4a37-b789-7400834e0189'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:55.380267 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '526aaf60dd6c42808951c5f48f8fabd2', 'request_id': 'req-6f042893-11b8-4a37-b789-7400834e0189'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:55.396176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '526aaf60dd6c42808951c5f48f8fabd2', 'request_id': 'req-6f042893-11b8-4a37-b789-7400834e0189'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:55.409316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] The token cache is being invalidate because project 526aaf60dd6c42808951c5f48f8fabd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:55.409588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:55.410249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6f042893-11b8-4a37-b789-7400834e0189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:55.415045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1123/2251] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:55 2022] DELETE /identity/v3/projects/526aaf60dd6c42808951c5f48f8fabd2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:55.417249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7a0bb-b46c-43ac-ac46-af9719ea54d1 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:55.417528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7a0bb-b46c-43ac-ac46-af9719ea54d1 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:55.417795 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7a0bb-b46c-43ac-ac46-af9719ea54d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:55.458349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-aba7a0bb-b46c-43ac-ac46-af9719ea54d1 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:55.460024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-aba7a0bb-b46c-43ac-ac46-af9719ea54d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:55.561965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1129/2252] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:55.570419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b06d3b-f826-45b9-8a90-7b6fed40cee3 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:55.570751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b06d3b-f826-45b9-8a90-7b6fed40cee3 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:55.571025 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b06d3b-f826-45b9-8a90-7b6fed40cee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:55.593124 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-31b06d3b-f826-45b9-8a90-7b6fed40cee3 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:55.594358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-31b06d3b-f826-45b9-8a90-7b6fed40cee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:55.604020 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1124/2253] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:55.608789 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7a0bb-b46c-43ac-ac46-af9719ea54d1 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:55.625509 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7a0bb-b46c-43ac-ac46-af9719ea54d1 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:55.631631 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212dfad-7ad1-4c36-8c53-bbf1bebda2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:55.632524 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212dfad-7ad1-4c36-8c53-bbf1bebda2bf admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:55.633569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212dfad-7ad1-4c36-8c53-bbf1bebda2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:55.634028 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212dfad-7ad1-4c36-8c53-bbf1bebda2bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:55.635136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212dfad-7ad1-4c36-8c53-bbf1bebda2bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:12:55.635709 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212dfad-7ad1-4c36-8c53-bbf1bebda2bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:55.637748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212dfad-7ad1-4c36-8c53-bbf1bebda2bf admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:55.658865 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1130/2254] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:12:55.663261 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b06d3b-f826-45b9-8a90-7b6fed40cee3 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:55.677833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b06d3b-f826-45b9-8a90-7b6fed40cee3 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:55.689623 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afdb5b-567d-4a47-8b92-cc476a03751f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:55.694210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afdb5b-567d-4a47-8b92-cc476a03751f admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:55.694519 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afdb5b-567d-4a47-8b92-cc476a03751f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:55.694801 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afdb5b-567d-4a47-8b92-cc476a03751f admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:55.695723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afdb5b-567d-4a47-8b92-cc476a03751f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:55.697793 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afdb5b-567d-4a47-8b92-cc476a03751f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:55.763814 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1125/2255] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:55 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:55.767669 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212dfad-7ad1-4c36-8c53-bbf1bebda2bf admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:55.791787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212dfad-7ad1-4c36-8c53-bbf1bebda2bf admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:55.850542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:55.916374 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da568b-b41e-4938-b4ff-88bbefee0317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:55.917621 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da568b-b41e-4938-b4ff-88bbefee0317 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:55.917621 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da568b-b41e-4938-b4ff-88bbefee0317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:55.917621 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da568b-b41e-4938-b4ff-88bbefee0317 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:55.918170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da568b-b41e-4938-b4ff-88bbefee0317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:55.920166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da568b-b41e-4938-b4ff-88bbefee0317 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:55.958692 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1131/2256] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:55 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:12:55.963351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afdb5b-567d-4a47-8b92-cc476a03751f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:55.981301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afdb5b-567d-4a47-8b92-cc476a03751f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:56.008545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f39d3-e752-41ed-a155-97831dc3aee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:56.009177 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f39d3-e752-41ed-a155-97831dc3aee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.009452 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f39d3-e752-41ed-a155-97831dc3aee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.009739 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f39d3-e752-41ed-a155-97831dc3aee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.010390 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f39d3-e752-41ed-a155-97831dc3aee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:56.013625 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f39d3-e752-41ed-a155-97831dc3aee3 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:56.016146 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:56.030543 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1126/2257] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:56.034541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da568b-b41e-4938-b4ff-88bbefee0317 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:56.048928 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da568b-b41e-4938-b4ff-88bbefee0317 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:56.056387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ac680-3d80-40cb-9555-1093363b5cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:56.057490 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ac680-3d80-40cb-9555-1093363b5cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.057771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ac680-3d80-40cb-9555-1093363b5cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.058465 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ac680-3d80-40cb-9555-1093363b5cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.059897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ac680-3d80-40cb-9555-1093363b5cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:56.066603 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ac680-3d80-40cb-9555-1093363b5cc4 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:56.079905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1132/2258] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:56.084130 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f39d3-e752-41ed-a155-97831dc3aee3 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:56.095560 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f39d3-e752-41ed-a155-97831dc3aee3 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:56.101678 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c631e-3b68-46b7-9bfe-6edca5722361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:56.102368 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c631e-3b68-46b7-9bfe-6edca5722361 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.102651 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c631e-3b68-46b7-9bfe-6edca5722361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.102954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c631e-3b68-46b7-9bfe-6edca5722361 admin admin] PATH_INFO: `/v3/projects/163bf317317942878e84fdab867179ca/users/b802c47d3b9c40d78366e8d5f681261a/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.120683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c631e-3b68-46b7-9bfe-6edca5722361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163bf317317942878e84fdab867179ca, user_id=b802c47d3b9c40d78366e8d5f681261a, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:56.124569 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c631e-3b68-46b7-9bfe-6edca5722361 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:56.153764 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1127/2259] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:56 2022] PUT /identity/v3/projects/163bf317317942878e84fdab867179ca/users/b802c47d3b9c40d78366e8d5f681261a/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:56.167539 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3045a4-ef29-41cb-bf69-dfaded00c258 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.168344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3045a4-ef29-41cb-bf69-dfaded00c258 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.168717 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3045a4-ef29-41cb-bf69-dfaded00c258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.199842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ce3045a4-ef29-41cb-bf69-dfaded00c258 None None] MFA Rules not processed for user `b802c47d3b9c40d78366e8d5f681261a`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:56.201259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ce3045a4-ef29-41cb-bf69-dfaded00c258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:56.273169 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1133/2260] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:56.277789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c631e-3b68-46b7-9bfe-6edca5722361 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:56.344030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c631e-3b68-46b7-9bfe-6edca5722361 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:56.349617 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f76606-ff9f-4ac0-a18c-5f4aa6a28d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:56.350238 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f76606-ff9f-4ac0-a18c-5f4aa6a28d3b admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.350571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f76606-ff9f-4ac0-a18c-5f4aa6a28d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.351015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f76606-ff9f-4ac0-a18c-5f4aa6a28d3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.351645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f76606-ff9f-4ac0-a18c-5f4aa6a28d3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:56.353781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f76606-ff9f-4ac0-a18c-5f4aa6a28d3b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:56.385070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1128/2261] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:56 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:56.386245 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3045a4-ef29-41cb-bf69-dfaded00c258 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:56.398478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3045a4-ef29-41cb-bf69-dfaded00c258 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:56.479696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac84cfa-43e6-43e4-b4bc-cdfb464fbad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:56.480318 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac84cfa-43e6-43e4-b4bc-cdfb464fbad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.480602 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac84cfa-43e6-43e4-b4bc-cdfb464fbad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.481370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac84cfa-43e6-43e4-b4bc-cdfb464fbad1 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.481763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac84cfa-43e6-43e4-b4bc-cdfb464fbad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:56.484766 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac84cfa-43e6-43e4-b4bc-cdfb464fbad1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:56.515715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1134/2262] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:56 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:12:56.517239 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f76606-ff9f-4ac0-a18c-5f4aa6a28d3b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:56.528598 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f76606-ff9f-4ac0-a18c-5f4aa6a28d3b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:56.534280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7e8a9-beaf-4041-8e99-edbb0bb8802b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:56.534899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7e8a9-beaf-4041-8e99-edbb0bb8802b admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.535219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7e8a9-beaf-4041-8e99-edbb0bb8802b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.535474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7e8a9-beaf-4041-8e99-edbb0bb8802b admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.536126 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7e8a9-beaf-4041-8e99-edbb0bb8802b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:56.537815 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7e8a9-beaf-4041-8e99-edbb0bb8802b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:56.555018 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1129/2263] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:12:56.558788 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac84cfa-43e6-43e4-b4bc-cdfb464fbad1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:56.569276 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac84cfa-43e6-43e4-b4bc-cdfb464fbad1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:56.574306 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9953c2-4220-4006-929e-f531683e49be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:56.574970 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9953c2-4220-4006-929e-f531683e49be admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.575338 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9953c2-4220-4006-929e-f531683e49be admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.575618 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9953c2-4220-4006-929e-f531683e49be admin admin] PATH_INFO: `/v3/projects/460c178bfb6345f895d2944e6d63bbd7/users/412250cd0ab34a13b836d1e29047e7c3/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.598373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9953c2-4220-4006-929e-f531683e49be admin admin] RBAC: Authorizing `identity:create_grant(project_id=460c178bfb6345f895d2944e6d63bbd7, user_id=412250cd0ab34a13b836d1e29047e7c3, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:56.601368 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9953c2-4220-4006-929e-f531683e49be admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:56.613981 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:56.620713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1135/2264] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:56 2022] PUT /identity/v3/projects/460c178bfb6345f895d2944e6d63bbd7/users/412250cd0ab34a13b836d1e29047e7c3/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:56.624981 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7e8a9-beaf-4041-8e99-edbb0bb8802b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:56.641657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7e8a9-beaf-4041-8e99-edbb0bb8802b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:56.722752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682b8bd-c096-40f1-9ba2-effda22a605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:56.723497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682b8bd-c096-40f1-9ba2-effda22a605e admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.723823 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682b8bd-c096-40f1-9ba2-effda22a605e admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.724102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682b8bd-c096-40f1-9ba2-effda22a605e admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.724758 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682b8bd-c096-40f1-9ba2-effda22a605e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:56.731102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682b8bd-c096-40f1-9ba2-effda22a605e admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:56.745796 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1130/2265] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:56.750750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9953c2-4220-4006-929e-f531683e49be admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:56.769755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9953c2-4220-4006-929e-f531683e49be admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:56.779934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde6ce3-5dc1-4949-8cbe-6995412d635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:56.780685 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde6ce3-5dc1-4949-8cbe-6995412d635e admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.781019 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde6ce3-5dc1-4949-8cbe-6995412d635e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.781347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde6ce3-5dc1-4949-8cbe-6995412d635e admin admin] PATH_INFO: `/v3/projects/460c178bfb6345f895d2944e6d63bbd7/users/412250cd0ab34a13b836d1e29047e7c3/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.802917 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde6ce3-5dc1-4949-8cbe-6995412d635e admin admin] RBAC: Authorizing `identity:create_grant(project_id=460c178bfb6345f895d2944e6d63bbd7, user_id=412250cd0ab34a13b836d1e29047e7c3, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:56.809755 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde6ce3-5dc1-4949-8cbe-6995412d635e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:56.825042 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1136/2266] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:56 2022] PUT /identity/v3/projects/460c178bfb6345f895d2944e6d63bbd7/users/412250cd0ab34a13b836d1e29047e7c3/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:56.836941 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753452e-1e34-4197-9079-639632e144b4 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:56.837301 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753452e-1e34-4197-9079-639632e144b4 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:56.837586 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753452e-1e34-4197-9079-639632e144b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:56.869607 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-0753452e-1e34-4197-9079-639632e144b4 None None] MFA Rules not processed for user `412250cd0ab34a13b836d1e29047e7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:56.870923 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-0753452e-1e34-4197-9079-639632e144b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:56.922427 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:56.983915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1131/2267] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:56.994716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde6ce3-5dc1-4949-8cbe-6995412d635e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:56.999008 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-dbde6ce3-5dc1-4949-8cbe-6995412d635e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:57.093027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde6ce3-5dc1-4949-8cbe-6995412d635e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:57.099167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1395e-7821-4ff1-933c-c3aab220dea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:57.099950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1395e-7821-4ff1-933c-c3aab220dea0 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:57.100379 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1395e-7821-4ff1-933c-c3aab220dea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:57.100798 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1395e-7821-4ff1-933c-c3aab220dea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:57.111892 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1395e-7821-4ff1-933c-c3aab220dea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:57.113652 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1395e-7821-4ff1-933c-c3aab220dea0 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:57.128455 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1137/2268] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:57.220448 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753452e-1e34-4197-9079-639632e144b4 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:57.229862 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753452e-1e34-4197-9079-639632e144b4 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:57.234723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f326072-f5d2-461e-8fbe-5598d522e7bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:57.234899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f326072-f5d2-461e-8fbe-5598d522e7bc service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:57.235089 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f326072-f5d2-461e-8fbe-5598d522e7bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:57.235364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f326072-f5d2-461e-8fbe-5598d522e7bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:57.309608 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f326072-f5d2-461e-8fbe-5598d522e7bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:57.318505 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f326072-f5d2-461e-8fbe-5598d522e7bc service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:57.325618 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1132/2269] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:12:57.346251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1395e-7821-4ff1-933c-c3aab220dea0 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:57.347928 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-0eb1395e-7821-4ff1-933c-c3aab220dea0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:57.421079 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1395e-7821-4ff1-933c-c3aab220dea0 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:57.425905 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:57.426559 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:57.426834 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:57.427140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] PATH_INFO: `/v3/users/e868bb74b33f467380562aec01a3d3e9` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:57.428283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e868bb74b33f467380562aec01a3d3e9)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:57.430220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:57.491601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e868bb74b33f467380562aec01a3d3e9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.492249 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e868bb74b33f467380562aec01a3d3e9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.504527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e868bb74b33f467380562aec01a3d3e9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.512826 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e868bb74b33f467380562aec01a3d3e9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.513581 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-9eb1ebb1-6bf8-4a83-af91-9003e4402040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e868bb74b33f467380562aec01a3d3e9'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.526574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1138/2270] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:12:57 2022] DELETE /identity/v3/users/e868bb74b33f467380562aec01a3d3e9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:57.704795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f326072-f5d2-461e-8fbe-5598d522e7bc service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:57.716187 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f326072-f5d2-461e-8fbe-5598d522e7bc service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:57.786457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:57.787132 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:57.787415 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:57.787687 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] PATH_INFO: `/v3/projects/55bc75ee717c43b7811b0ab11c647f99` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:57.789496 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=55bc75ee717c43b7811b0ab11c647f99)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:57.791554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:57.844539 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55bc75ee717c43b7811b0ab11c647f99', 'request_id': 'req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.851219 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55bc75ee717c43b7811b0ab11c647f99', 'request_id': 'req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.860872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55bc75ee717c43b7811b0ab11c647f99', 'request_id': 'req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.865999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:57.875461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] The token cache is being invalidate because project 55bc75ee717c43b7811b0ab11c647f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:12:57.875461 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.876692 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-93ff1bc1-47a5-4530-99f2-24fa53b96ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:12:57.879538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1133/2271] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:12:57 2022] DELETE /identity/v3/projects/55bc75ee717c43b7811b0ab11c647f99 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:57.885926 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85ba91-0df5-462c-afdd-cc0f21111b8d None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:57.886293 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85ba91-0df5-462c-afdd-cc0f21111b8d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:57.886568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85ba91-0df5-462c-afdd-cc0f21111b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:57.900574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:12:57.909099 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-ca85ba91-0df5-462c-afdd-cc0f21111b8d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:57.910950 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-ca85ba91-0df5-462c-afdd-cc0f21111b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:57.971358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1139/2272] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:57.978905 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372f60-4978-4093-9f5d-246b634f7a16 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:57.979224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372f60-4978-4093-9f5d-246b634f7a16 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:57.979497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372f60-4978-4093-9f5d-246b634f7a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:58.000443 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-f3372f60-4978-4093-9f5d-246b634f7a16 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:58.001677 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-f3372f60-4978-4093-9f5d-246b634f7a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:58.009966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1134/2273] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:58.014991 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85ba91-0df5-462c-afdd-cc0f21111b8d None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:58.025136 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85ba91-0df5-462c-afdd-cc0f21111b8d None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:58.030599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350648b-7284-4380-b903-a75327ef8a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:58.031260 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350648b-7284-4380-b903-a75327ef8a83 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:58.031540 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350648b-7284-4380-b903-a75327ef8a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:58.031816 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350648b-7284-4380-b903-a75327ef8a83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:58.034479 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350648b-7284-4380-b903-a75327ef8a83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:12:58.034794 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350648b-7284-4380-b903-a75327ef8a83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:58.036500 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350648b-7284-4380-b903-a75327ef8a83 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:58.050172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1140/2274] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:12:58.054682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3372f60-4978-4093-9f5d-246b634f7a16 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:58.063980 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3372f60-4978-4093-9f5d-246b634f7a16 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:58.068869 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd51b17-f46a-4004-9d88-074e8da2cc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:58.069504 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd51b17-f46a-4004-9d88-074e8da2cc39 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:58.069836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd51b17-f46a-4004-9d88-074e8da2cc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:58.070110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd51b17-f46a-4004-9d88-074e8da2cc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:58.070749 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd51b17-f46a-4004-9d88-074e8da2cc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:58.072724 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd51b17-f46a-4004-9d88-074e8da2cc39 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:58.103658 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1135/2275] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:12:58 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:58.107718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350648b-7284-4380-b903-a75327ef8a83 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:58.117712 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350648b-7284-4380-b903-a75327ef8a83 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:58.176257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8751-4d75-4556-b38e-a07b16c37a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:58.176922 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8751-4d75-4556-b38e-a07b16c37a8e admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:58.177244 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8751-4d75-4556-b38e-a07b16c37a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:58.177510 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8751-4d75-4556-b38e-a07b16c37a8e admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:58.178156 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b8751-4d75-4556-b38e-a07b16c37a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:58.180144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b8751-4d75-4556-b38e-a07b16c37a8e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:58.204597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1141/2276] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:12:58 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:12:58.205747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd51b17-f46a-4004-9d88-074e8da2cc39 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:58.216259 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd51b17-f46a-4004-9d88-074e8da2cc39 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:58.221768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b4479-28f2-403a-b9e9-9dea18264f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:58.222302 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b4479-28f2-403a-b9e9-9dea18264f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:58.222588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b4479-28f2-403a-b9e9-9dea18264f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:58.222859 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b4479-28f2-403a-b9e9-9dea18264f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:58.223533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b4479-28f2-403a-b9e9-9dea18264f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:58.225150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b4479-28f2-403a-b9e9-9dea18264f0d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:58.235966 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1136/2277] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:12:58.240045 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8751-4d75-4556-b38e-a07b16c37a8e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:58.250873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8751-4d75-4556-b38e-a07b16c37a8e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:58.255768 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e417563-328c-4d32-b01b-5680fd9f35d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:58.256358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e417563-328c-4d32-b01b-5680fd9f35d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:58.256634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e417563-328c-4d32-b01b-5680fd9f35d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:58.256902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e417563-328c-4d32-b01b-5680fd9f35d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:58.257538 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e417563-328c-4d32-b01b-5680fd9f35d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:58.259422 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e417563-328c-4d32-b01b-5680fd9f35d6 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:58.270134 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1142/2278] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:12:58.274353 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b4479-28f2-403a-b9e9-9dea18264f0d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:58.284922 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b4479-28f2-403a-b9e9-9dea18264f0d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:58.290010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb1b36-d764-4eb7-b737-a951c5224d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:58.290663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb1b36-d764-4eb7-b737-a951c5224d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:58.290947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb1b36-d764-4eb7-b737-a951c5224d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:58.291264 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb1b36-d764-4eb7-b737-a951c5224d48 admin admin] PATH_INFO: `/v3/projects/7ac6ccae08d64b5e97805897acdb9aa4/users/f7c55684a73c4fd49a629012f2d4d661/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:58.306546 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb1b36-d764-4eb7-b737-a951c5224d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac6ccae08d64b5e97805897acdb9aa4, user_id=f7c55684a73c4fd49a629012f2d4d661, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:58.309385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb1b36-d764-4eb7-b737-a951c5224d48 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:58.323483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1137/2279] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:12:58 2022] PUT /identity/v3/projects/7ac6ccae08d64b5e97805897acdb9aa4/users/f7c55684a73c4fd49a629012f2d4d661/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:12:58.328714 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54aa70-dd1d-47e0-b2af-25853db55aa4 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:58.328998 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54aa70-dd1d-47e0-b2af-25853db55aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:58.329302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54aa70-dd1d-47e0-b2af-25853db55aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:58.348355 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-cd54aa70-dd1d-47e0-b2af-25853db55aa4 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:12:58.349574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-cd54aa70-dd1d-47e0-b2af-25853db55aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:58.411290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1143/2280] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:12:58.421770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb1b36-d764-4eb7-b737-a951c5224d48 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:12:58.423545 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5bcb1b36-d764-4eb7-b737-a951c5224d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:12:58.480667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb1b36-d764-4eb7-b737-a951c5224d48 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:12:58.484883 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02b3ea-a44a-49e3-832a-655a809cfcd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:12:58.485493 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02b3ea-a44a-49e3-832a-655a809cfcd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:12:58.485770 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02b3ea-a44a-49e3-832a-655a809cfcd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:12:58.486041 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02b3ea-a44a-49e3-832a-655a809cfcd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:12:58.496023 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02b3ea-a44a-49e3-832a-655a809cfcd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:12:58.497454 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02b3ea-a44a-49e3-832a-655a809cfcd4 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:12:58.508034 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1138/2281] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:03.271699 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f7552-c4c9-420b-8d37-9b375299f39f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:03.272233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f7552-c4c9-420b-8d37-9b375299f39f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:03.272233 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f7552-c4c9-420b-8d37-9b375299f39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:03.314833 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:03.319307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-543f7552-c4c9-420b-8d37-9b375299f39f None None] MFA Rules not processed for user `f7c55684a73c4fd49a629012f2d4d661`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:03.321177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-543f7552-c4c9-420b-8d37-9b375299f39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:03.396107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1144/2282] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:03.410893 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02b3ea-a44a-49e3-832a-655a809cfcd4 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:03.413381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-3c02b3ea-a44a-49e3-832a-655a809cfcd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:03.554914 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02b3ea-a44a-49e3-832a-655a809cfcd4 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:03.560983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa747498-8770-415a-8693-be7ae3bb95c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a758611cf8d94e8ca8aad6a75bef5e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:03.561709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa747498-8770-415a-8693-be7ae3bb95c4 service nova] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:03.562060 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa747498-8770-415a-8693-be7ae3bb95c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:03.562436 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa747498-8770-415a-8693-be7ae3bb95c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:03.572166 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa747498-8770-415a-8693-be7ae3bb95c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:03.573771 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa747498-8770-415a-8693-be7ae3bb95c4 service nova] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:03.584915 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1139/2283] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:05.162086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc9e32-b18d-4afc-bee8-0f289f14851e None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:05.162634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc9e32-b18d-4afc-bee8-0f289f14851e None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:05.162634 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc9e32-b18d-4afc-bee8-0f289f14851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:05.182307 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-4abc9e32-b18d-4afc-bee8-0f289f14851e None None] MFA Rules not processed for user `f7c55684a73c4fd49a629012f2d4d661`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:05.183655 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-4abc9e32-b18d-4afc-bee8-0f289f14851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:05.191930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1145/2284] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:13.579882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:15.177475 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=29 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:37.146176 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa747498-8770-415a-8693-be7ae3bb95c4 service nova] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:37.148185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-fa747498-8770-415a-8693-be7ae3bb95c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:37.162494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa747498-8770-415a-8693-be7ae3bb95c4 service nova] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:37.167825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:37.168467 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:37.168978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:37.168978 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] PATH_INFO: `/v3/users/b802c47d3b9c40d78366e8d5f681261a` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:37.170230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b802c47d3b9c40d78366e8d5f681261a)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:37.172011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:37.229662 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b802c47d3b9c40d78366e8d5f681261a'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:37.230234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b802c47d3b9c40d78366e8d5f681261a'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:37.244049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b802c47d3b9c40d78366e8d5f681261a'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:37.251205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b802c47d3b9c40d78366e8d5f681261a'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:37.260721 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b802c47d3b9c40d78366e8d5f681261a'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:37.267683 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1140/2285] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:13:37 2022] DELETE /identity/v3/users/b802c47d3b9c40d78366e8d5f681261a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:13:37.272378 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20a3b-dc13-44d9-92c7-5d00e3e7c34f None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:37.272885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20a3b-dc13-44d9-92c7-5d00e3e7c34f None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:37.273000 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20a3b-dc13-44d9-92c7-5d00e3e7c34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:37.292967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-33b20a3b-dc13-44d9-92c7-5d00e3e7c34f None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:37.294176 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-33b20a3b-dc13-44d9-92c7-5d00e3e7c34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:37.367037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1146/2286] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:13:37.377410 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:37.379603 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:37.455700 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb34a6-5fb6-449c-be64-f27346e6a681 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:37.461515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e509682-60c5-45eb-ae83-7743376f6c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:37.462134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e509682-60c5-45eb-ae83-7743376f6c1e service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:37.462406 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e509682-60c5-45eb-ae83-7743376f6c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:37.462669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e509682-60c5-45eb-ae83-7743376f6c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:37.476432 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e509682-60c5-45eb-ae83-7743376f6c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:37.476962 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e509682-60c5-45eb-ae83-7743376f6c1e service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:37.489659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1141/2287] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:37.900238 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20a3b-dc13-44d9-92c7-5d00e3e7c34f None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:37.902099 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-33b20a3b-dc13-44d9-92c7-5d00e3e7c34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:37.915882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20a3b-dc13-44d9-92c7-5d00e3e7c34f None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:37.921964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:37.922575 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:37.923453 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:37.923716 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] PATH_INFO: `/v3/projects/163bf317317942878e84fdab867179ca` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:37.924881 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=163bf317317942878e84fdab867179ca)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:37.926813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:37.981655 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163bf317317942878e84fdab867179ca', 'request_id': 'req-80621702-27f7-47f9-ac51-c811519768ee'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:37.991612 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163bf317317942878e84fdab867179ca', 'request_id': 'req-80621702-27f7-47f9-ac51-c811519768ee'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:37.999011 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163bf317317942878e84fdab867179ca', 'request_id': 'req-80621702-27f7-47f9-ac51-c811519768ee'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.010614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] The token cache is being invalidate because project 163bf317317942878e84fdab867179ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:13:38.010897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.011604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.013096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1147/2288] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:13:37 2022] DELETE /identity/v3/projects/163bf317317942878e84fdab867179ca => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:38.017655 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e509682-60c5-45eb-ae83-7743376f6c1e service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:38.029086 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e509682-60c5-45eb-ae83-7743376f6c1e service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:38.103483 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:38.104110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:38.104381 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:38.104806 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] PATH_INFO: `/v3/users/412250cd0ab34a13b836d1e29047e7c3` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:38.105947 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=412250cd0ab34a13b836d1e29047e7c3)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:38.107961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:38.128316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:38.174308 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '412250cd0ab34a13b836d1e29047e7c3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.175153 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '412250cd0ab34a13b836d1e29047e7c3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.188658 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '412250cd0ab34a13b836d1e29047e7c3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.196107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '412250cd0ab34a13b836d1e29047e7c3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.205185 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-c66db0d2-2386-42cf-a220-ea6962b912b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '412250cd0ab34a13b836d1e29047e7c3'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.212376 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1142/2289] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:13:38 2022] DELETE /identity/v3/users/412250cd0ab34a13b836d1e29047e7c3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:38.547171 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:38.549297 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:38.568084 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80621702-27f7-47f9-ac51-c811519768ee admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:38.630404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:38.631024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:38.631289 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:38.631599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] PATH_INFO: `/v3/projects/460c178bfb6345f895d2944e6d63bbd7` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:38.633096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=460c178bfb6345f895d2944e6d63bbd7)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:38.635039 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:38.674614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:38.694835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '460c178bfb6345f895d2944e6d63bbd7', 'request_id': 'req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.712227 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '460c178bfb6345f895d2944e6d63bbd7', 'request_id': 'req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.719362 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '460c178bfb6345f895d2944e6d63bbd7', 'request_id': 'req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.731790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] The token cache is being invalidate because project 460c178bfb6345f895d2944e6d63bbd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:13:38.732087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.732767 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-16668c7a-aef7-4d75-ad21-74fb53f3a5ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:38.734285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1148/2290] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:13:38 2022] DELETE /identity/v3/projects/460c178bfb6345f895d2944e6d63bbd7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:38.742971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c44a17-b74c-4005-9fdd-0db594ca6a46 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:38.743575 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c44a17-b74c-4005-9fdd-0db594ca6a46 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:38.744012 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c44a17-b74c-4005-9fdd-0db594ca6a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:38.766918 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-03c44a17-b74c-4005-9fdd-0db594ca6a46 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:38.768247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-03c44a17-b74c-4005-9fdd-0db594ca6a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:38.845885 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1143/2291] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:38.854172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89317606-27e0-4399-876c-6ccc4c9a5824 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:38.854460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89317606-27e0-4399-876c-6ccc4c9a5824 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:38.854733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89317606-27e0-4399-876c-6ccc4c9a5824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:38.876706 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-89317606-27e0-4399-876c-6ccc4c9a5824 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:38.877939 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-89317606-27e0-4399-876c-6ccc4c9a5824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:38.887194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1149/2292] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:13:38.892439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c44a17-b74c-4005-9fdd-0db594ca6a46 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:38.902457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c44a17-b74c-4005-9fdd-0db594ca6a46 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:38.908562 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af300eab-cef4-4ee6-b060-f6757fa4daac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:38.909170 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af300eab-cef4-4ee6-b060-f6757fa4daac admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:38.909447 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af300eab-cef4-4ee6-b060-f6757fa4daac admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:38.909714 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af300eab-cef4-4ee6-b060-f6757fa4daac admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:38.910364 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af300eab-cef4-4ee6-b060-f6757fa4daac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:13:38.910667 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af300eab-cef4-4ee6-b060-f6757fa4daac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:38.912812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af300eab-cef4-4ee6-b060-f6757fa4daac admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:38.929241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1144/2293] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:13:38.934902 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89317606-27e0-4399-876c-6ccc4c9a5824 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:38.944822 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89317606-27e0-4399-876c-6ccc4c9a5824 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:38.949813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7d68-c6b9-4a62-987e-5f680bafaba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:38.950515 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7d68-c6b9-4a62-987e-5f680bafaba1 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:38.950864 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7d68-c6b9-4a62-987e-5f680bafaba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:38.951326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7d68-c6b9-4a62-987e-5f680bafaba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:38.952025 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b7d68-c6b9-4a62-987e-5f680bafaba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:38.954507 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b7d68-c6b9-4a62-987e-5f680bafaba1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:38.987447 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1150/2294] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:13:38 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:38.992188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af300eab-cef4-4ee6-b060-f6757fa4daac admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.003156 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af300eab-cef4-4ee6-b060-f6757fa4daac admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.070373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fec1c0-a775-441e-9a82-6e297b232380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.070373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fec1c0-a775-441e-9a82-6e297b232380 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.071439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fec1c0-a775-441e-9a82-6e297b232380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.071439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fec1c0-a775-441e-9a82-6e297b232380 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.072423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fec1c0-a775-441e-9a82-6e297b232380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.074110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fec1c0-a775-441e-9a82-6e297b232380 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.100435 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1145/2295] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:13:38 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:39.104769 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7d68-c6b9-4a62-987e-5f680bafaba1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.117483 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7d68-c6b9-4a62-987e-5f680bafaba1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.122624 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca590308-2348-4e15-9417-e4190a5878ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.123261 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca590308-2348-4e15-9417-e4190a5878ec admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.123532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca590308-2348-4e15-9417-e4190a5878ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.123806 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca590308-2348-4e15-9417-e4190a5878ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.124458 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca590308-2348-4e15-9417-e4190a5878ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.126700 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca590308-2348-4e15-9417-e4190a5878ec admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.142047 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1151/2296] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:13:39.146277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fec1c0-a775-441e-9a82-6e297b232380 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.157630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fec1c0-a775-441e-9a82-6e297b232380 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.162593 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c6197-829a-4f54-841f-4b23a84a049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.163230 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c6197-829a-4f54-841f-4b23a84a049f admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.163506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c6197-829a-4f54-841f-4b23a84a049f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.163773 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c6197-829a-4f54-841f-4b23a84a049f admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.164412 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c6197-829a-4f54-841f-4b23a84a049f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.166006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c6197-829a-4f54-841f-4b23a84a049f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.171917 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:39.183798 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1146/2297] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:39.184744 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca590308-2348-4e15-9417-e4190a5878ec admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.196031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca590308-2348-4e15-9417-e4190a5878ec admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.200925 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d89d01-5ea6-494e-bddc-4f3a48831943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.201579 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d89d01-5ea6-494e-bddc-4f3a48831943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.201857 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d89d01-5ea6-494e-bddc-4f3a48831943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.202130 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d89d01-5ea6-494e-bddc-4f3a48831943 admin admin] PATH_INFO: `/v3/projects/518179db59e04ede8724d30bc41a2ba7/users/8d4ca29a741b4dfd9a5dd18010623dae/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.217522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d89d01-5ea6-494e-bddc-4f3a48831943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518179db59e04ede8724d30bc41a2ba7, user_id=8d4ca29a741b4dfd9a5dd18010623dae, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.220752 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d89d01-5ea6-494e-bddc-4f3a48831943 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.233923 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1152/2298] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:39 2022] PUT /identity/v3/projects/518179db59e04ede8724d30bc41a2ba7/users/8d4ca29a741b4dfd9a5dd18010623dae/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:13:39.245247 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3210b8-f155-4d4e-b2f5-8f52ff047f4b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.245544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3210b8-f155-4d4e-b2f5-8f52ff047f4b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.245825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3210b8-f155-4d4e-b2f5-8f52ff047f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.276175 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-2e3210b8-f155-4d4e-b2f5-8f52ff047f4b None None] MFA Rules not processed for user `8d4ca29a741b4dfd9a5dd18010623dae`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:39.277539 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-2e3210b8-f155-4d4e-b2f5-8f52ff047f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:39.350835 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1147/2299] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:13:39.357348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d89d01-5ea6-494e-bddc-4f3a48831943 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.429967 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d89d01-5ea6-494e-bddc-4f3a48831943 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.435919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2279c7d-25dc-40b8-8e8f-95e9749db6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.436611 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2279c7d-25dc-40b8-8e8f-95e9749db6fa admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.436884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2279c7d-25dc-40b8-8e8f-95e9749db6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.437158 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2279c7d-25dc-40b8-8e8f-95e9749db6fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.437859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2279c7d-25dc-40b8-8e8f-95e9749db6fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.439914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2279c7d-25dc-40b8-8e8f-95e9749db6fa admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.468057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1153/2300] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:13:39 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:39.473704 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3210b8-f155-4d4e-b2f5-8f52ff047f4b None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.489438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3210b8-f155-4d4e-b2f5-8f52ff047f4b None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.563160 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b83a72-6478-4c9a-be92-8e50c3ee9a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.564090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b83a72-6478-4c9a-be92-8e50c3ee9a03 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.564581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b83a72-6478-4c9a-be92-8e50c3ee9a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.565028 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b83a72-6478-4c9a-be92-8e50c3ee9a03 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.566057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b83a72-6478-4c9a-be92-8e50c3ee9a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.569243 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b83a72-6478-4c9a-be92-8e50c3ee9a03 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.597891 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1148/2301] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:13:39 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:39.602793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2279c7d-25dc-40b8-8e8f-95e9749db6fa admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.614675 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2279c7d-25dc-40b8-8e8f-95e9749db6fa admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.619632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eaea6b-93f2-4da6-9274-cd74bea6cbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.620237 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eaea6b-93f2-4da6-9274-cd74bea6cbeb admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.620514 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eaea6b-93f2-4da6-9274-cd74bea6cbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.620778 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eaea6b-93f2-4da6-9274-cd74bea6cbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.621424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eaea6b-93f2-4da6-9274-cd74bea6cbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.623044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eaea6b-93f2-4da6-9274-cd74bea6cbeb admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.635661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1154/2302] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:39.640246 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b83a72-6478-4c9a-be92-8e50c3ee9a03 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.651200 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b83a72-6478-4c9a-be92-8e50c3ee9a03 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.656117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcd055-cdef-406c-b854-bc8951de655f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.656781 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcd055-cdef-406c-b854-bc8951de655f admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.657127 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcd055-cdef-406c-b854-bc8951de655f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.657407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcd055-cdef-406c-b854-bc8951de655f admin admin] PATH_INFO: `/v3/projects/9501dc6b95754910a06830a04e8e972e/users/25dedd440ce244e9b3b8bcaa978aeced/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.675723 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcd055-cdef-406c-b854-bc8951de655f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9501dc6b95754910a06830a04e8e972e, user_id=25dedd440ce244e9b3b8bcaa978aeced, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.678477 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcd055-cdef-406c-b854-bc8951de655f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.695657 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1149/2303] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:39 2022] PUT /identity/v3/projects/9501dc6b95754910a06830a04e8e972e/users/25dedd440ce244e9b3b8bcaa978aeced/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:39.700252 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eaea6b-93f2-4da6-9274-cd74bea6cbeb admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.715991 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eaea6b-93f2-4da6-9274-cd74bea6cbeb admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.739235 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:39.784848 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea01559-88cf-4785-8de3-06fcc5ff8323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.785456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01559-88cf-4785-8de3-06fcc5ff8323 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.785764 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01559-88cf-4785-8de3-06fcc5ff8323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.786035 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01559-88cf-4785-8de3-06fcc5ff8323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.786686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea01559-88cf-4785-8de3-06fcc5ff8323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.788315 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea01559-88cf-4785-8de3-06fcc5ff8323 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.799044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1155/2304] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:39.803419 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcd055-cdef-406c-b854-bc8951de655f admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.814583 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcd055-cdef-406c-b854-bc8951de655f admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:39.819402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d557f-8887-4b24-8d49-12cc7d2ddb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:39.820026 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d557f-8887-4b24-8d49-12cc7d2ddb12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.820311 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d557f-8887-4b24-8d49-12cc7d2ddb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.820584 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d557f-8887-4b24-8d49-12cc7d2ddb12 admin admin] PATH_INFO: `/v3/projects/9501dc6b95754910a06830a04e8e972e/users/25dedd440ce244e9b3b8bcaa978aeced/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.822550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d557f-8887-4b24-8d49-12cc7d2ddb12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9501dc6b95754910a06830a04e8e972e, user_id=25dedd440ce244e9b3b8bcaa978aeced, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:39.825242 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d557f-8887-4b24-8d49-12cc7d2ddb12 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:39.836150 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1150/2305] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:39 2022] PUT /identity/v3/projects/9501dc6b95754910a06830a04e8e972e/users/25dedd440ce244e9b3b8bcaa978aeced/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:39.848365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ad256-36cf-4e4d-b51e-d7d2ce15c012 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:39.848649 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ad256-36cf-4e4d-b51e-d7d2ce15c012 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:39.848919 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ad256-36cf-4e4d-b51e-d7d2ce15c012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:39.879544 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-016ad256-36cf-4e4d-b51e-d7d2ce15c012 None None] MFA Rules not processed for user `25dedd440ce244e9b3b8bcaa978aeced`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:39.881391 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-016ad256-36cf-4e4d-b51e-d7d2ce15c012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:39.957811 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1156/2306] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:39.970854 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d557f-8887-4b24-8d49-12cc7d2ddb12 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:39.974171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-db9d557f-8887-4b24-8d49-12cc7d2ddb12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:40.033257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:40.048987 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d557f-8887-4b24-8d49-12cc7d2ddb12 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:40.052992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff809928-a11b-4315-bdf5-95865dda2410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:40.053613 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff809928-a11b-4315-bdf5-95865dda2410 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:40.053888 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff809928-a11b-4315-bdf5-95865dda2410 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:40.054167 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff809928-a11b-4315-bdf5-95865dda2410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:40.064457 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff809928-a11b-4315-bdf5-95865dda2410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:40.065937 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff809928-a11b-4315-bdf5-95865dda2410 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:40.076288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1151/2307] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:40.109777 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ad256-36cf-4e4d-b51e-d7d2ce15c012 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:40.184787 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ad256-36cf-4e4d-b51e-d7d2ce15c012 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:40.190302 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:40.190945 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:40.191283 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:40.191546 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] PATH_INFO: `/v3/users/8d4ca29a741b4dfd9a5dd18010623dae` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:40.192748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4ca29a741b4dfd9a5dd18010623dae)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:40.194661 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:40.245968 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4ca29a741b4dfd9a5dd18010623dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.246678 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4ca29a741b4dfd9a5dd18010623dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.248126 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:40.262083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4ca29a741b4dfd9a5dd18010623dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.269227 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4ca29a741b4dfd9a5dd18010623dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.269995 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4ca29a741b4dfd9a5dd18010623dae'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.287599 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1157/2308] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:13:40 2022] DELETE /identity/v3/users/8d4ca29a741b4dfd9a5dd18010623dae => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:13:40.293134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c81e-9f45-4fb9-b49c-9e35e3f1835c None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:40.293652 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c81e-9f45-4fb9-b49c-9e35e3f1835c None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:40.293797 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c81e-9f45-4fb9-b49c-9e35e3f1835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:40.314120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-1112c81e-9f45-4fb9-b49c-9e35e3f1835c None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:40.315366 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1112c81e-9f45-4fb9-b49c-9e35e3f1835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:40.377672 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1152/2309] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:40.387682 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:40.389095 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:40.448616 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658ddff-0d50-4dc4-8a59-b4911fdb47e0 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:40.453056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cc18b-6a82-4cde-ada7-07a056724e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:40.453711 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cc18b-6a82-4cde-ada7-07a056724e8c service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:40.454504 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cc18b-6a82-4cde-ada7-07a056724e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:40.454771 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cc18b-6a82-4cde-ada7-07a056724e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:40.464517 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cc18b-6a82-4cde-ada7-07a056724e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:40.465961 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cc18b-6a82-4cde-ada7-07a056724e8c service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:40.484419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1158/2310] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:40.828017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112c81e-9f45-4fb9-b49c-9e35e3f1835c None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:40.830179 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-1112c81e-9f45-4fb9-b49c-9e35e3f1835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:40.844343 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112c81e-9f45-4fb9-b49c-9e35e3f1835c None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:40.850611 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:40.851298 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:40.851572 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:40.851899 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] PATH_INFO: `/v3/projects/518179db59e04ede8724d30bc41a2ba7` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:40.853248 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=518179db59e04ede8724d30bc41a2ba7)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:40.855313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:40.913734 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518179db59e04ede8724d30bc41a2ba7', 'request_id': 'req-9b0618cc-15f5-490a-b6fb-9d28422be517'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.924394 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518179db59e04ede8724d30bc41a2ba7', 'request_id': 'req-9b0618cc-15f5-490a-b6fb-9d28422be517'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.934015 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518179db59e04ede8724d30bc41a2ba7', 'request_id': 'req-9b0618cc-15f5-490a-b6fb-9d28422be517'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.948245 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] The token cache is being invalidate because project 518179db59e04ede8724d30bc41a2ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:13:40.948594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.949423 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:40.951056 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1153/2311] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:13:40 2022] DELETE /identity/v3/projects/518179db59e04ede8724d30bc41a2ba7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:40.955718 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cc18b-6a82-4cde-ada7-07a056724e8c service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:40.965734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cc18b-6a82-4cde-ada7-07a056724e8c service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:41.027730 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:41.028372 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:41.028652 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:41.028932 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] PATH_INFO: `/v3/users/25dedd440ce244e9b3b8bcaa978aeced` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:41.030124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=25dedd440ce244e9b3b8bcaa978aeced)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:41.032217 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:41.082488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25dedd440ce244e9b3b8bcaa978aeced'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.083175 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25dedd440ce244e9b3b8bcaa978aeced'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.104356 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25dedd440ce244e9b3b8bcaa978aeced'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.111243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:41.116914 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25dedd440ce244e9b3b8bcaa978aeced'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.118246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-71107c71-35e2-4c25-a1a0-b470d9d06b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25dedd440ce244e9b3b8bcaa978aeced'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.135722 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1159/2312] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:13:40 2022] DELETE /identity/v3/users/25dedd440ce244e9b3b8bcaa978aeced => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:41.473265 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:41.474730 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:41.493232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0618cc-15f5-490a-b6fb-9d28422be517 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:41.560330 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:41.560866 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:41.561140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:41.561408 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] PATH_INFO: `/v3/projects/9501dc6b95754910a06830a04e8e972e` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:41.562535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9501dc6b95754910a06830a04e8e972e)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:41.564491 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:41.576115 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:41.615557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9501dc6b95754910a06830a04e8e972e', 'request_id': 'req-7d146062-d453-4ae0-8f09-3e80006ebea9'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.624538 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9501dc6b95754910a06830a04e8e972e', 'request_id': 'req-7d146062-d453-4ae0-8f09-3e80006ebea9'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.632983 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9501dc6b95754910a06830a04e8e972e', 'request_id': 'req-7d146062-d453-4ae0-8f09-3e80006ebea9'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.644337 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] The token cache is being invalidate because project 9501dc6b95754910a06830a04e8e972e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:13:41.644629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.645274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-7d146062-d453-4ae0-8f09-3e80006ebea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:41.646767 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1154/2313] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:13:41 2022] DELETE /identity/v3/projects/9501dc6b95754910a06830a04e8e972e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:41.651808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd66fc-8fe5-493c-b2a1-5843d8996858 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:41.652086 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd66fc-8fe5-493c-b2a1-5843d8996858 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:41.652352 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd66fc-8fe5-493c-b2a1-5843d8996858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:41.677166 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-c2cd66fc-8fe5-493c-b2a1-5843d8996858 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:41.678373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c2cd66fc-8fe5-493c-b2a1-5843d8996858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:41.744421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1160/2314] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:41.752441 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b31f7f-669d-4272-bcf3-30ac0688c807 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:41.752716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b31f7f-669d-4272-bcf3-30ac0688c807 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:41.752994 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b31f7f-669d-4272-bcf3-30ac0688c807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:41.773754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-56b31f7f-669d-4272-bcf3-30ac0688c807 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:41.775013 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-56b31f7f-669d-4272-bcf3-30ac0688c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:41.784116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1155/2315] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:41.788573 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd66fc-8fe5-493c-b2a1-5843d8996858 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:41.799713 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd66fc-8fe5-493c-b2a1-5843d8996858 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:41.804488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87644b-108a-4ecb-ac5c-2492773b82c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:41.805105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87644b-108a-4ecb-ac5c-2492773b82c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:41.805383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87644b-108a-4ecb-ac5c-2492773b82c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:41.805655 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87644b-108a-4ecb-ac5c-2492773b82c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:41.806305 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87644b-108a-4ecb-ac5c-2492773b82c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:13:41.806763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87644b-108a-4ecb-ac5c-2492773b82c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:41.809495 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87644b-108a-4ecb-ac5c-2492773b82c1 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:41.822099 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1161/2316] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:13:41.826456 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b31f7f-669d-4272-bcf3-30ac0688c807 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:41.836674 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b31f7f-669d-4272-bcf3-30ac0688c807 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:41.842136 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7add5-d479-4a09-8274-4072ec5f14c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:41.842745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7add5-d479-4a09-8274-4072ec5f14c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:41.843049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7add5-d479-4a09-8274-4072ec5f14c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:41.843321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7add5-d479-4a09-8274-4072ec5f14c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:41.843960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7add5-d479-4a09-8274-4072ec5f14c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:41.846508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7add5-d479-4a09-8274-4072ec5f14c9 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:41.880351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1156/2317] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:13:41 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:41.886288 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87644b-108a-4ecb-ac5c-2492773b82c1 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:41.905257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87644b-108a-4ecb-ac5c-2492773b82c1 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:41.992641 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140751da-e4ff-4f21-88a2-d00fbe78f44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:41.993605 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140751da-e4ff-4f21-88a2-d00fbe78f44e admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:41.993605 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140751da-e4ff-4f21-88a2-d00fbe78f44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:41.993913 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140751da-e4ff-4f21-88a2-d00fbe78f44e admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:41.994383 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140751da-e4ff-4f21-88a2-d00fbe78f44e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:41.996371 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140751da-e4ff-4f21-88a2-d00fbe78f44e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.019964 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1162/2318] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:13:41 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:42.024531 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7add5-d479-4a09-8274-4072ec5f14c9 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.036557 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7add5-d479-4a09-8274-4072ec5f14c9 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.041429 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d4c0e-c08f-4367-9d4b-adc661f65b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.042030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d4c0e-c08f-4367-9d4b-adc661f65b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.042316 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d4c0e-c08f-4367-9d4b-adc661f65b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.042588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d4c0e-c08f-4367-9d4b-adc661f65b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.043255 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d4c0e-c08f-4367-9d4b-adc661f65b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.044867 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d4c0e-c08f-4367-9d4b-adc661f65b41 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.059742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1157/2319] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:42.064277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140751da-e4ff-4f21-88a2-d00fbe78f44e admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.079321 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140751da-e4ff-4f21-88a2-d00fbe78f44e admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.091193 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebb360-d4af-4b56-955c-773e0ca23fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.092167 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebb360-d4af-4b56-955c-773e0ca23fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.092633 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebb360-d4af-4b56-955c-773e0ca23fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.093105 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebb360-d4af-4b56-955c-773e0ca23fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.094073 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebb360-d4af-4b56-955c-773e0ca23fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.095989 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebb360-d4af-4b56-955c-773e0ca23fa8 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.107481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1163/2320] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:13:42.112064 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d4c0e-c08f-4367-9d4b-adc661f65b41 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.122515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d4c0e-c08f-4367-9d4b-adc661f65b41 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.127559 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c3fc8-08cd-4018-8fc0-377d81f31dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.128214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c3fc8-08cd-4018-8fc0-377d81f31dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.128494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c3fc8-08cd-4018-8fc0-377d81f31dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.128790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c3fc8-08cd-4018-8fc0-377d81f31dd5 admin admin] PATH_INFO: `/v3/projects/2f15b9cacd8e456d9bbb2da4a10f2af6/users/0a3cd291c5424db499d622b46a2c1668/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.146221 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c3fc8-08cd-4018-8fc0-377d81f31dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f15b9cacd8e456d9bbb2da4a10f2af6, user_id=0a3cd291c5424db499d622b46a2c1668, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.148949 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c3fc8-08cd-4018-8fc0-377d81f31dd5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.164622 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1158/2321] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:42 2022] PUT /identity/v3/projects/2f15b9cacd8e456d9bbb2da4a10f2af6/users/0a3cd291c5424db499d622b46a2c1668/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:42.177332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4830e-98f2-499e-bd3a-c950785a0d64 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.177653 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4830e-98f2-499e-bd3a-c950785a0d64 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.177921 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4830e-98f2-499e-bd3a-c950785a0d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.215847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-d6b4830e-98f2-499e-bd3a-c950785a0d64 None None] MFA Rules not processed for user `0a3cd291c5424db499d622b46a2c1668`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:42.217729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-d6b4830e-98f2-499e-bd3a-c950785a0d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:42.262632 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:42.295961 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1164/2322] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:42.300533 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c3fc8-08cd-4018-8fc0-377d81f31dd5 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.370344 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c3fc8-08cd-4018-8fc0-377d81f31dd5 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.374961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb41647-52f7-4282-bf77-2cc3e45d9331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.376107 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb41647-52f7-4282-bf77-2cc3e45d9331 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.376383 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb41647-52f7-4282-bf77-2cc3e45d9331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.376645 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb41647-52f7-4282-bf77-2cc3e45d9331 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.377277 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb41647-52f7-4282-bf77-2cc3e45d9331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.379204 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb41647-52f7-4282-bf77-2cc3e45d9331 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.413641 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1159/2323] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:13:42 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:42.420424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4830e-98f2-499e-bd3a-c950785a0d64 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.435541 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4830e-98f2-499e-bd3a-c950785a0d64 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.510710 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d92742-9f2e-49d6-98ab-529aaaf6f4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.511365 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d92742-9f2e-49d6-98ab-529aaaf6f4bf admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.511662 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d92742-9f2e-49d6-98ab-529aaaf6f4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.511928 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d92742-9f2e-49d6-98ab-529aaaf6f4bf admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.512557 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d92742-9f2e-49d6-98ab-529aaaf6f4bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.514566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d92742-9f2e-49d6-98ab-529aaaf6f4bf admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.544435 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1165/2324] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:13:42 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:13:42.546297 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb41647-52f7-4282-bf77-2cc3e45d9331 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.558471 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb41647-52f7-4282-bf77-2cc3e45d9331 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.563824 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539165b3-1120-4415-a4e9-8ee34ed17e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.564438 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539165b3-1120-4415-a4e9-8ee34ed17e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.564718 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539165b3-1120-4415-a4e9-8ee34ed17e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.564988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539165b3-1120-4415-a4e9-8ee34ed17e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.565643 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539165b3-1120-4415-a4e9-8ee34ed17e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.567319 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539165b3-1120-4415-a4e9-8ee34ed17e8e admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.579985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1160/2325] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:42.585640 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d92742-9f2e-49d6-98ab-529aaaf6f4bf admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.599337 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d92742-9f2e-49d6-98ab-529aaaf6f4bf admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.605111 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548945a-2319-4a41-ae2e-797a5665a7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.605748 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548945a-2319-4a41-ae2e-797a5665a7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.606019 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548945a-2319-4a41-ae2e-797a5665a7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.606287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548945a-2319-4a41-ae2e-797a5665a7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.606930 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548945a-2319-4a41-ae2e-797a5665a7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.608705 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548945a-2319-4a41-ae2e-797a5665a7f4 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.620141 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1166/2326] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:42.624926 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539165b3-1120-4415-a4e9-8ee34ed17e8e admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.636929 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539165b3-1120-4415-a4e9-8ee34ed17e8e admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.643090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c68b3-0d1b-49b9-a14b-f3155e0b2588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.643822 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c68b3-0d1b-49b9-a14b-f3155e0b2588 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.644045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c68b3-0d1b-49b9-a14b-f3155e0b2588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.644335 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c68b3-0d1b-49b9-a14b-f3155e0b2588 admin admin] PATH_INFO: `/v3/projects/aa8723e01944453f9c5afe56d9302b67/users/a954d4e045364b26b446dad0f87495fc/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.647807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:42.665131 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c68b3-0d1b-49b9-a14b-f3155e0b2588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8723e01944453f9c5afe56d9302b67, user_id=a954d4e045364b26b446dad0f87495fc, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.668342 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c68b3-0d1b-49b9-a14b-f3155e0b2588 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.681953 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1161/2327] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:42 2022] PUT /identity/v3/projects/aa8723e01944453f9c5afe56d9302b67/users/a954d4e045364b26b446dad0f87495fc/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:42.692399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fae972-1413-4eca-93b8-d3fe9e27a4f3 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.692676 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fae972-1413-4eca-93b8-d3fe9e27a4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.692946 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fae972-1413-4eca-93b8-d3fe9e27a4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.726031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-e2fae972-1413-4eca-93b8-d3fe9e27a4f3 None None] MFA Rules not processed for user `a954d4e045364b26b446dad0f87495fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:42.727327 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-e2fae972-1413-4eca-93b8-d3fe9e27a4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:42.801377 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1167/2328] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:42.806241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c68b3-0d1b-49b9-a14b-f3155e0b2588 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.870629 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c68b3-0d1b-49b9-a14b-f3155e0b2588 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.875647 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a17035-04da-447b-8d39-a5e1bfe067d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.876193 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a17035-04da-447b-8d39-a5e1bfe067d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.876478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a17035-04da-447b-8d39-a5e1bfe067d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.876737 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a17035-04da-447b-8d39-a5e1bfe067d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.877361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a17035-04da-447b-8d39-a5e1bfe067d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.879313 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a17035-04da-447b-8d39-a5e1bfe067d2 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:42.909125 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1162/2329] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:13:42 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:42.913215 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fae972-1413-4eca-93b8-d3fe9e27a4f3 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:42.924445 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fae972-1413-4eca-93b8-d3fe9e27a4f3 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:42.976679 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:42.989173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d295a6-6930-426e-b81e-ca57729ba657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:42.989904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d295a6-6930-426e-b81e-ca57729ba657 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:42.990210 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d295a6-6930-426e-b81e-ca57729ba657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:42.990477 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d295a6-6930-426e-b81e-ca57729ba657 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:42.991140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d295a6-6930-426e-b81e-ca57729ba657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:42.993056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d295a6-6930-426e-b81e-ca57729ba657 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:43.017209 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1168/2330] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:13:42 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:43.021254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a17035-04da-447b-8d39-a5e1bfe067d2 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:43.032414 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a17035-04da-447b-8d39-a5e1bfe067d2 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:43.037371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ec8c5-f0b1-4190-8ea0-2a88bfa890cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:43.037991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ec8c5-f0b1-4190-8ea0-2a88bfa890cf admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:43.038270 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ec8c5-f0b1-4190-8ea0-2a88bfa890cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:43.038544 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ec8c5-f0b1-4190-8ea0-2a88bfa890cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:43.039214 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ec8c5-f0b1-4190-8ea0-2a88bfa890cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:43.040813 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ec8c5-f0b1-4190-8ea0-2a88bfa890cf admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:43.052949 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1163/2331] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:43.057527 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d295a6-6930-426e-b81e-ca57729ba657 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:43.068370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d295a6-6930-426e-b81e-ca57729ba657 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:43.073151 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99529b6c-bcaa-467c-8301-34c0e632bd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:43.073797 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99529b6c-bcaa-467c-8301-34c0e632bd7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:43.074073 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99529b6c-bcaa-467c-8301-34c0e632bd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:43.074341 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99529b6c-bcaa-467c-8301-34c0e632bd7a admin admin] PATH_INFO: `/v3/projects/39ed0f91c03a42e89c5ab18ef00cb33f/users/81df424e5c6e459d92752f7975b87126/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:43.091404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99529b6c-bcaa-467c-8301-34c0e632bd7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ed0f91c03a42e89c5ab18ef00cb33f, user_id=81df424e5c6e459d92752f7975b87126, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:43.094093 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99529b6c-bcaa-467c-8301-34c0e632bd7a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:43.109727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1169/2332] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:43 2022] PUT /identity/v3/projects/39ed0f91c03a42e89c5ab18ef00cb33f/users/81df424e5c6e459d92752f7975b87126/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:43.114664 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ec8c5-f0b1-4190-8ea0-2a88bfa890cf admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:43.124365 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ec8c5-f0b1-4190-8ea0-2a88bfa890cf admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:43.188585 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859155a-aeec-4266-acaa-f3b24130589d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:43.189106 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859155a-aeec-4266-acaa-f3b24130589d admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:43.189385 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859155a-aeec-4266-acaa-f3b24130589d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:43.189656 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859155a-aeec-4266-acaa-f3b24130589d admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:43.190327 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859155a-aeec-4266-acaa-f3b24130589d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:43.191961 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859155a-aeec-4266-acaa-f3b24130589d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:43.201863 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1164/2333] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:13:43.206643 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99529b6c-bcaa-467c-8301-34c0e632bd7a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:43.218248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99529b6c-bcaa-467c-8301-34c0e632bd7a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:43.223366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e54043-18d0-4df1-8b8f-282930eb4333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:43.224138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e54043-18d0-4df1-8b8f-282930eb4333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:43.224466 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e54043-18d0-4df1-8b8f-282930eb4333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:43.224707 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e54043-18d0-4df1-8b8f-282930eb4333 admin admin] PATH_INFO: `/v3/projects/39ed0f91c03a42e89c5ab18ef00cb33f/users/81df424e5c6e459d92752f7975b87126/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:43.226644 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e54043-18d0-4df1-8b8f-282930eb4333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ed0f91c03a42e89c5ab18ef00cb33f, user_id=81df424e5c6e459d92752f7975b87126, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:43.229334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e54043-18d0-4df1-8b8f-282930eb4333 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:43.240419 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1170/2334] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:43 2022] PUT /identity/v3/projects/39ed0f91c03a42e89c5ab18ef00cb33f/users/81df424e5c6e459d92752f7975b87126/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:43.251676 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb84f7-bbc1-41f2-abc9-f9ed2a207b1d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:43.251960 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb84f7-bbc1-41f2-abc9-f9ed2a207b1d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:43.252233 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb84f7-bbc1-41f2-abc9-f9ed2a207b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:43.281663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-b8eb84f7-bbc1-41f2-abc9-f9ed2a207b1d None None] MFA Rules not processed for user `81df424e5c6e459d92752f7975b87126`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:43.283070 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b8eb84f7-bbc1-41f2-abc9-f9ed2a207b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:43.328522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:43.354585 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1165/2335] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:13:43.366087 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e54043-18d0-4df1-8b8f-282930eb4333 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:43.367842 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-63e54043-18d0-4df1-8b8f-282930eb4333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:43.428444 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e54043-18d0-4df1-8b8f-282930eb4333 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:43.432135 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40005e-8ce6-41ec-bbc8-b513c6da78f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:43.432740 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40005e-8ce6-41ec-bbc8-b513c6da78f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:43.433022 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40005e-8ce6-41ec-bbc8-b513c6da78f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:43.433287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40005e-8ce6-41ec-bbc8-b513c6da78f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:43.443560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40005e-8ce6-41ec-bbc8-b513c6da78f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:43.444911 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40005e-8ce6-41ec-bbc8-b513c6da78f6 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:43.454629 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1171/2336] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:44.020421 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb84f7-bbc1-41f2-abc9-f9ed2a207b1d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:44.022228 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-b8eb84f7-bbc1-41f2-abc9-f9ed2a207b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:44.044698 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb84f7-bbc1-41f2-abc9-f9ed2a207b1d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:44.051590 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200bb16-a2ce-450a-bdd4-f467721ddf78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:44.052514 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200bb16-a2ce-450a-bdd4-f467721ddf78 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:44.052942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200bb16-a2ce-450a-bdd4-f467721ddf78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:44.053420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200bb16-a2ce-450a-bdd4-f467721ddf78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:44.128075 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200bb16-a2ce-450a-bdd4-f467721ddf78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:44.129535 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200bb16-a2ce-450a-bdd4-f467721ddf78 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:44.139755 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1166/2337] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:48.342194 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40005e-8ce6-41ec-bbc8-b513c6da78f6 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:48.358688 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40005e-8ce6-41ec-bbc8-b513c6da78f6 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:48.367750 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10510a-fd9f-4f22-85ab-fc675080b853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:48.368349 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10510a-fd9f-4f22-85ab-fc675080b853 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:48.368635 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10510a-fd9f-4f22-85ab-fc675080b853 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:48.368912 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10510a-fd9f-4f22-85ab-fc675080b853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:48.427522 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10510a-fd9f-4f22-85ab-fc675080b853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:48.428884 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10510a-fd9f-4f22-85ab-fc675080b853 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:48.439469 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1172/2338] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:50.730716 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200bb16-a2ce-450a-bdd4-f467721ddf78 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:50.807134 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200bb16-a2ce-450a-bdd4-f467721ddf78 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:50.811665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:50.815117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:50.815117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:50.815117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] PATH_INFO: `/v3/users/0a3cd291c5424db499d622b46a2c1668` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:50.815117 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3cd291c5424db499d622b46a2c1668)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:50.816669 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:50.872416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3cd291c5424db499d622b46a2c1668'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:50.873063 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3cd291c5424db499d622b46a2c1668'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:50.887198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3cd291c5424db499d622b46a2c1668'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:50.901615 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3cd291c5424db499d622b46a2c1668'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:50.910828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3cd291c5424db499d622b46a2c1668'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:50.912398 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:50.919494 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1167/2339] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:13:50 2022] DELETE /identity/v3/users/0a3cd291c5424db499d622b46a2c1668 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:50.924498 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538b792-4651-4d50-9739-ed058735f701 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:50.925225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538b792-4651-4d50-9739-ed058735f701 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:50.925225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538b792-4651-4d50-9739-ed058735f701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:50.944344 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6538b792-4651-4d50-9739-ed058735f701 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:50.945568 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6538b792-4651-4d50-9739-ed058735f701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:50.988607 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:51.013173 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1173/2340] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:13:51.023988 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:51.025812 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:51.088524 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd54fc-7434-48ca-8614-f6c3c694c42d admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:51.092163 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477f82b-83da-41e2-ac2f-09d82c8ce959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:51.092765 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477f82b-83da-41e2-ac2f-09d82c8ce959 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:51.093045 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477f82b-83da-41e2-ac2f-09d82c8ce959 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:51.093320 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477f82b-83da-41e2-ac2f-09d82c8ce959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:51.103484 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477f82b-83da-41e2-ac2f-09d82c8ce959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:51.104930 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477f82b-83da-41e2-ac2f-09d82c8ce959 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:51.115257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1168/2341] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:51.436259 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538b792-4651-4d50-9739-ed058735f701 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:51.437815 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6538b792-4651-4d50-9739-ed058735f701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:51.452424 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538b792-4651-4d50-9739-ed058735f701 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:51.459044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:51.459471 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:51.459808 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:51.460142 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] PATH_INFO: `/v3/projects/2f15b9cacd8e456d9bbb2da4a10f2af6` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:51.461481 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f15b9cacd8e456d9bbb2da4a10f2af6)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:51.463901 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:51.521331 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f15b9cacd8e456d9bbb2da4a10f2af6', 'request_id': 'req-a1b0dedf-9c33-4a61-b7fd-a17b94880024'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.531501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f15b9cacd8e456d9bbb2da4a10f2af6', 'request_id': 'req-a1b0dedf-9c33-4a61-b7fd-a17b94880024'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.539115 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f15b9cacd8e456d9bbb2da4a10f2af6', 'request_id': 'req-a1b0dedf-9c33-4a61-b7fd-a17b94880024'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.553582 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] The token cache is being invalidate because project 2f15b9cacd8e456d9bbb2da4a10f2af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:13:51.553877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.554542 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.556108 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1174/2342] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:13:51 2022] DELETE /identity/v3/projects/2f15b9cacd8e456d9bbb2da4a10f2af6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:13:51.560495 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477f82b-83da-41e2-ac2f-09d82c8ce959 service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:51.570420 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477f82b-83da-41e2-ac2f-09d82c8ce959 service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:51.631203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:51.631736 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:51.632009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:51.632550 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] PATH_INFO: `/v3/users/a954d4e045364b26b446dad0f87495fc` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:51.633878 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a954d4e045364b26b446dad0f87495fc)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:51.635977 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:51.688091 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a954d4e045364b26b446dad0f87495fc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.688742 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a954d4e045364b26b446dad0f87495fc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.701224 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a954d4e045364b26b446dad0f87495fc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.708795 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a954d4e045364b26b446dad0f87495fc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.716601 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a954d4e045364b26b446dad0f87495fc'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:51.723340 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1169/2343] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:13:51 2022] DELETE /identity/v3/users/a954d4e045364b26b446dad0f87495fc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:52.008715 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:52.010304 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:52.024358 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0dedf-9c33-4a61-b7fd-a17b94880024 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:52.083892 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:52.084528 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:52.084803 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:52.085076 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] PATH_INFO: `/v3/projects/aa8723e01944453f9c5afe56d9302b67` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:52.086195 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8723e01944453f9c5afe56d9302b67)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:52.088256 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:52.135862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8723e01944453f9c5afe56d9302b67', 'request_id': 'req-324f012e-e20b-4bc5-945d-e78544b4c7ed'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.144328 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8723e01944453f9c5afe56d9302b67', 'request_id': 'req-324f012e-e20b-4bc5-945d-e78544b4c7ed'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.150897 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8723e01944453f9c5afe56d9302b67', 'request_id': 'req-324f012e-e20b-4bc5-945d-e78544b4c7ed'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.157392 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:52.162656 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] The token cache is being invalidate because project aa8723e01944453f9c5afe56d9302b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:13:52.163031 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.163965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.166065 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1175/2344] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:13:52 2022] DELETE /identity/v3/projects/aa8723e01944453f9c5afe56d9302b67 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:52.172006 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:52.173816 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:52.186523 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9b50d-73fb-4ce7-b6fe-6279e2014962 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:52.203119 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:52.244272 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:52.244898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:52.245174 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:52.245446 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] PATH_INFO: `/v3/users/81df424e5c6e459d92752f7975b87126` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:52.246565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] RBAC: Authorizing `identity:delete_user(user_id=81df424e5c6e459d92752f7975b87126)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:52.248522 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:52.294141 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81df424e5c6e459d92752f7975b87126'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.294790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81df424e5c6e459d92752f7975b87126'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.306971 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81df424e5c6e459d92752f7975b87126'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.314919 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81df424e5c6e459d92752f7975b87126'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.323053 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-492fa1e5-05c5-4764-986d-13bb5a14203e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81df424e5c6e459d92752f7975b87126'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.329090 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1170/2345] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:13:52 2022] DELETE /identity/v3/users/81df424e5c6e459d92752f7975b87126 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:52.643512 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:52.645001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:52.659762 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f012e-e20b-4bc5-945d-e78544b4c7ed admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:52.717149 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:52.717780 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:52.718069 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:52.718323 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] PATH_INFO: `/v3/projects/39ed0f91c03a42e89c5ab18ef00cb33f` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:52.720051 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ed0f91c03a42e89c5ab18ef00cb33f)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:52.721956 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:52.777489 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ed0f91c03a42e89c5ab18ef00cb33f', 'request_id': 'req-bf30c281-5ad3-4799-9cb0-0a47238015b5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.787459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ed0f91c03a42e89c5ab18ef00cb33f', 'request_id': 'req-bf30c281-5ad3-4799-9cb0-0a47238015b5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.795275 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ed0f91c03a42e89c5ab18ef00cb33f', 'request_id': 'req-bf30c281-5ad3-4799-9cb0-0a47238015b5'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.806285 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] The token cache is being invalidate because project 39ed0f91c03a42e89c5ab18ef00cb33f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:13:52.806571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.807248 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-bf30c281-5ad3-4799-9cb0-0a47238015b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:13:52.808734 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1176/2346] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:13:52 2022] DELETE /identity/v3/projects/39ed0f91c03a42e89c5ab18ef00cb33f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:52.814942 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27d3d-890e-4574-bbad-94f33812630b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:52.815256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27d3d-890e-4574-bbad-94f33812630b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:52.815530 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27d3d-890e-4574-bbad-94f33812630b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:52.835336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-def27d3d-890e-4574-bbad-94f33812630b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:52.836542 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-def27d3d-890e-4574-bbad-94f33812630b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:52.897392 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1171/2347] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:52.907688 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c1601-778e-492c-8d07-87e1400fda1d None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:52.908060 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c1601-778e-492c-8d07-87e1400fda1d None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:52.908329 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c1601-778e-492c-8d07-87e1400fda1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:52.928534 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-f02c1601-778e-492c-8d07-87e1400fda1d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:52.929654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-f02c1601-778e-492c-8d07-87e1400fda1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:52.937586 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1177/2348] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:52.945017 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def27d3d-890e-4574-bbad-94f33812630b None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:52.954634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def27d3d-890e-4574-bbad-94f33812630b None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:52.959199 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc6558-399a-4c79-878b-5eef4db10b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:52.959791 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc6558-399a-4c79-878b-5eef4db10b94 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:52.960061 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc6558-399a-4c79-878b-5eef4db10b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:52.960325 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc6558-399a-4c79-878b-5eef4db10b94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:52.960955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc6558-399a-4c79-878b-5eef4db10b94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:13:52.961276 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc6558-399a-4c79-878b-5eef4db10b94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:52.962856 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc6558-399a-4c79-878b-5eef4db10b94 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:52.975051 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1172/2349] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:13:52.979569 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c1601-778e-492c-8d07-87e1400fda1d None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:52.989654 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c1601-778e-492c-8d07-87e1400fda1d None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:52.994328 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a34b7-9c38-4bb1-a798-8d1dac646c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:52.994954 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a34b7-9c38-4bb1-a798-8d1dac646c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:52.995267 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a34b7-9c38-4bb1-a798-8d1dac646c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:52.995536 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a34b7-9c38-4bb1-a798-8d1dac646c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:52.996220 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a34b7-9c38-4bb1-a798-8d1dac646c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:52.998114 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a34b7-9c38-4bb1-a798-8d1dac646c08 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.026960 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1178/2350] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:13:52 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:53.033577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc6558-399a-4c79-878b-5eef4db10b94 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.047188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc6558-399a-4c79-878b-5eef4db10b94 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.071574 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:53.105116 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23008839-5f85-4df8-b475-42f0072727a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.105711 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23008839-5f85-4df8-b475-42f0072727a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.105985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23008839-5f85-4df8-b475-42f0072727a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.106249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23008839-5f85-4df8-b475-42f0072727a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.106859 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23008839-5f85-4df8-b475-42f0072727a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.108792 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23008839-5f85-4df8-b475-42f0072727a8 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.129565 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1173/2351] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:13:53 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:53.133977 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a34b7-9c38-4bb1-a798-8d1dac646c08 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.145906 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a34b7-9c38-4bb1-a798-8d1dac646c08 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.150552 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef902326-847d-4cdb-a804-e31b276afb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.151182 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef902326-847d-4cdb-a804-e31b276afb66 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.151462 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef902326-847d-4cdb-a804-e31b276afb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.151733 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef902326-847d-4cdb-a804-e31b276afb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.152376 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef902326-847d-4cdb-a804-e31b276afb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.153976 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef902326-847d-4cdb-a804-e31b276afb66 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.164565 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1179/2352] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:53.168618 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23008839-5f85-4df8-b475-42f0072727a8 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.178688 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23008839-5f85-4df8-b475-42f0072727a8 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.183361 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb32c87-7976-4dbc-94a3-2933d6d303c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.183956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32c87-7976-4dbc-94a3-2933d6d303c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.184229 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32c87-7976-4dbc-94a3-2933d6d303c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.184485 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32c87-7976-4dbc-94a3-2933d6d303c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.185118 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb32c87-7976-4dbc-94a3-2933d6d303c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.186702 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb32c87-7976-4dbc-94a3-2933d6d303c5 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.197515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1174/2353] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:53.201933 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef902326-847d-4cdb-a804-e31b276afb66 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.212572 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef902326-847d-4cdb-a804-e31b276afb66 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.217201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1a126-ee8d-43e9-873d-0061ae781915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.217843 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1a126-ee8d-43e9-873d-0061ae781915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.218122 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1a126-ee8d-43e9-873d-0061ae781915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.218394 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1a126-ee8d-43e9-873d-0061ae781915 admin admin] PATH_INFO: `/v3/projects/6341dd89bded4731a43045f695a1eed0/users/5b79923bf8d84d34ab337ae029af68c8/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.233478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1a126-ee8d-43e9-873d-0061ae781915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6341dd89bded4731a43045f695a1eed0, user_id=5b79923bf8d84d34ab337ae029af68c8, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.236222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1a126-ee8d-43e9-873d-0061ae781915 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.250190 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1180/2354] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:53 2022] PUT /identity/v3/projects/6341dd89bded4731a43045f695a1eed0/users/5b79923bf8d84d34ab337ae029af68c8/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:53.261828 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33707ff7-5e7c-4ed7-81a8-ac89bfb51f2a None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.262110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33707ff7-5e7c-4ed7-81a8-ac89bfb51f2a None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.262387 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33707ff7-5e7c-4ed7-81a8-ac89bfb51f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.291339 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-33707ff7-5e7c-4ed7-81a8-ac89bfb51f2a None None] MFA Rules not processed for user `5b79923bf8d84d34ab337ae029af68c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:53.292563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-33707ff7-5e7c-4ed7-81a8-ac89bfb51f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:53.356284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1175/2355] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:53.360952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1a126-ee8d-43e9-873d-0061ae781915 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.387639 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:53.426399 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1a126-ee8d-43e9-873d-0061ae781915 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.431179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354f8b8-28b3-4c81-9891-1830d584b145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.431835 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354f8b8-28b3-4c81-9891-1830d584b145 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.432118 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354f8b8-28b3-4c81-9891-1830d584b145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.432387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354f8b8-28b3-4c81-9891-1830d584b145 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.433037 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354f8b8-28b3-4c81-9891-1830d584b145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.434941 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354f8b8-28b3-4c81-9891-1830d584b145 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.462415 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1181/2356] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:13:53 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:13:53.466532 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33707ff7-5e7c-4ed7-81a8-ac89bfb51f2a None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.478033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33707ff7-5e7c-4ed7-81a8-ac89bfb51f2a None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.535879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38a902-9988-43f3-bece-3b9c449d13fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.536481 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a902-9988-43f3-bece-3b9c449d13fd admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.536759 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a902-9988-43f3-bece-3b9c449d13fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.537031 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a902-9988-43f3-bece-3b9c449d13fd admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.537665 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38a902-9988-43f3-bece-3b9c449d13fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.539628 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38a902-9988-43f3-bece-3b9c449d13fd admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.561844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1176/2357] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:13:53 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 19:13:53.565908 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354f8b8-28b3-4c81-9891-1830d584b145 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.576348 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354f8b8-28b3-4c81-9891-1830d584b145 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.580904 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067fa20-80ac-486f-9c0e-2906bee8d2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.581496 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067fa20-80ac-486f-9c0e-2906bee8d2ab admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.581772 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067fa20-80ac-486f-9c0e-2906bee8d2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.582038 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067fa20-80ac-486f-9c0e-2906bee8d2ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.582672 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067fa20-80ac-486f-9c0e-2906bee8d2ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.584279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067fa20-80ac-486f-9c0e-2906bee8d2ab admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.595367 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1182/2358] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:53.599567 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38a902-9988-43f3-bece-3b9c449d13fd admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.609709 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38a902-9988-43f3-bece-3b9c449d13fd admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.615253 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0d162-a1eb-4a8d-a77f-f3fd0a8dd101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.615910 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d162-a1eb-4a8d-a77f-f3fd0a8dd101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.616186 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d162-a1eb-4a8d-a77f-f3fd0a8dd101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.616460 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d162-a1eb-4a8d-a77f-f3fd0a8dd101 admin admin] PATH_INFO: `/v3/projects/6038c843cf3d4268ba8953ca36cb5ad0/users/e804b2ec2c7f4645a660a72720348316/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.632416 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d162-a1eb-4a8d-a77f-f3fd0a8dd101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6038c843cf3d4268ba8953ca36cb5ad0, user_id=e804b2ec2c7f4645a660a72720348316, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.635402 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d162-a1eb-4a8d-a77f-f3fd0a8dd101 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.645486 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:53.650946 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1177/2359] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:53 2022] PUT /identity/v3/projects/6038c843cf3d4268ba8953ca36cb5ad0/users/e804b2ec2c7f4645a660a72720348316/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:53.655143 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067fa20-80ac-486f-9c0e-2906bee8d2ab admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.665819 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067fa20-80ac-486f-9c0e-2906bee8d2ab admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.723452 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42ce8d-be9f-4a70-b24b-ab20e48cf29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.724062 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42ce8d-be9f-4a70-b24b-ab20e48cf29c admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.724337 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42ce8d-be9f-4a70-b24b-ab20e48cf29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.724604 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42ce8d-be9f-4a70-b24b-ab20e48cf29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.725244 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42ce8d-be9f-4a70-b24b-ab20e48cf29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.726862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42ce8d-be9f-4a70-b24b-ab20e48cf29c admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.743163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1183/2360] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:13:53.744468 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0d162-a1eb-4a8d-a77f-f3fd0a8dd101 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.757426 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0d162-a1eb-4a8d-a77f-f3fd0a8dd101 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.768290 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8df2a-0e06-419c-8434-39664179bcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.768972 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8df2a-0e06-419c-8434-39664179bcce admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.769249 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8df2a-0e06-419c-8434-39664179bcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.769526 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8df2a-0e06-419c-8434-39664179bcce admin admin] PATH_INFO: `/v3/projects/6038c843cf3d4268ba8953ca36cb5ad0/users/e804b2ec2c7f4645a660a72720348316/roles/5b9587b54a454ed5b8a7e02ed7071c8f` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.771585 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8df2a-0e06-419c-8434-39664179bcce admin admin] RBAC: Authorizing `identity:create_grant(project_id=6038c843cf3d4268ba8953ca36cb5ad0, user_id=e804b2ec2c7f4645a660a72720348316, role_id=5b9587b54a454ed5b8a7e02ed7071c8f)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:53.774256 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8df2a-0e06-419c-8434-39664179bcce admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:53.786727 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1178/2361] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:13:53 2022] PUT /identity/v3/projects/6038c843cf3d4268ba8953ca36cb5ad0/users/e804b2ec2c7f4645a660a72720348316/roles/5b9587b54a454ed5b8a7e02ed7071c8f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:13:53.797960 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93985-a14e-40f8-8699-9dc3fa8c30bd None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.798250 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93985-a14e-40f8-8699-9dc3fa8c30bd None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.798560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93985-a14e-40f8-8699-9dc3fa8c30bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:53.827795 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-02d93985-a14e-40f8-8699-9dc3fa8c30bd None None] MFA Rules not processed for user `e804b2ec2c7f4645a660a72720348316`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:13:53.830056 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-02d93985-a14e-40f8-8699-9dc3fa8c30bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:53.902107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1184/2362] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:13:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:13:53.914336 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8df2a-0e06-419c-8434-39664179bcce admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:53.916048 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-91e8df2a-0e06-419c-8434-39664179bcce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:13:53.973745 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8df2a-0e06-419c-8434-39664179bcce admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:53.977619 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf604fca-0b1e-4202-81a4-d4a3e0cce576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:53.978231 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf604fca-0b1e-4202-81a4-d4a3e0cce576 service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:53.978508 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf604fca-0b1e-4202-81a4-d4a3e0cce576 service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:53.978778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf604fca-0b1e-4202-81a4-d4a3e0cce576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:54.036375 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf604fca-0b1e-4202-81a4-d4a3e0cce576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:54.037515 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf604fca-0b1e-4202-81a4-d4a3e0cce576 service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:54.046694 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1179/2363] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:13:58.352877 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93985-a14e-40f8-8699-9dc3fa8c30bd None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:13:58.372938 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93985-a14e-40f8-8699-9dc3fa8c30bd None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:13:58.377601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b7880-8909-42ee-b34f-45ca0d613fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:13:58.377601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b7880-8909-42ee-b34f-45ca0d613fce service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:13:58.377601 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b7880-8909-42ee-b34f-45ca0d613fce service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:13:58.377918 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b7880-8909-42ee-b34f-45ca0d613fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:13:58.382846 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:13:58.414334 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b7880-8909-42ee-b34f-45ca0d613fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:13:58.415833 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b7880-8909-42ee-b34f-45ca0d613fce service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:13:58.426610 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1185/2364] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:13:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:14:03.468940 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf604fca-0b1e-4202-81a4-d4a3e0cce576 service cinder] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:03.610778 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf604fca-0b1e-4202-81a4-d4a3e0cce576 service cinder] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:03.618551 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:03.619842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:03.620148 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:03.620820 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] PATH_INFO: `/v3/users/5b79923bf8d84d34ab337ae029af68c8` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:03.622725 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b79923bf8d84d34ab337ae029af68c8)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:03.626630 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:03.643285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:03.695577 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b79923bf8d84d34ab337ae029af68c8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:03.696236 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b79923bf8d84d34ab337ae029af68c8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:03.710634 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b79923bf8d84d34ab337ae029af68c8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:03.718120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b79923bf8d84d34ab337ae029af68c8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:03.730553 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b79923bf8d84d34ab337ae029af68c8'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:03.745836 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1180/2365] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:14:03 2022] DELETE /identity/v3/users/5b79923bf8d84d34ab337ae029af68c8 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:14:03.751971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d8767-bf8a-4902-bbc8-b1397a4d3907 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:03.752952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d8767-bf8a-4902-bbc8-b1397a4d3907 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:03.752952 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d8767-bf8a-4902-bbc8-b1397a4d3907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:03.840083 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-349d8767-bf8a-4902-bbc8-b1397a4d3907 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:03.841560 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-349d8767-bf8a-4902-bbc8-b1397a4d3907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:03.926790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1186/2366] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:14:03.937789 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:03.939284 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:04.013500 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7125b-da91-4efe-8b31-4b2b41213117 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:04.017847 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa31e5-2978-4ff4-ac57-ee6a05f15f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:04.018506 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa31e5-2978-4ff4-ac57-ee6a05f15f6a service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:04.018790 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa31e5-2978-4ff4-ac57-ee6a05f15f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:04.019094 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa31e5-2978-4ff4-ac57-ee6a05f15f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:04.029747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa31e5-2978-4ff4-ac57-ee6a05f15f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:04.031211 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa31e5-2978-4ff4-ac57-ee6a05f15f6a service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:04.041624 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1181/2367] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:14:04.361440 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d8767-bf8a-4902-bbc8-b1397a4d3907 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:04.363593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-349d8767-bf8a-4902-bbc8-b1397a4d3907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:04.393428 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d8767-bf8a-4902-bbc8-b1397a4d3907 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:04.409124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:04.410366 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:04.410747 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:04.410940 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] PATH_INFO: `/v3/projects/6341dd89bded4731a43045f695a1eed0` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:04.412328 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6341dd89bded4731a43045f695a1eed0)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:04.415576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:04.489287 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6341dd89bded4731a43045f695a1eed0', 'request_id': 'req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.498793 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6341dd89bded4731a43045f695a1eed0', 'request_id': 'req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.508225 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6341dd89bded4731a43045f695a1eed0', 'request_id': 'req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.522370 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] The token cache is being invalidate because project 6341dd89bded4731a43045f695a1eed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:14:04.522688 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.523614 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.526927 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1187/2368] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:14:04 2022] DELETE /identity/v3/projects/6341dd89bded4731a43045f695a1eed0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:14:04.531597 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa31e5-2978-4ff4-ac57-ee6a05f15f6a service neutron] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:04.553054 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa31e5-2978-4ff4-ac57-ee6a05f15f6a service neutron] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:04.647223 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:04.647955 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:04.648329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:04.648924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] PATH_INFO: `/v3/users/e804b2ec2c7f4645a660a72720348316` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:04.650190 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e804b2ec2c7f4645a660a72720348316)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:04.652747 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:04.697653 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:04.749254 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e804b2ec2c7f4645a660a72720348316'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.750458 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e804b2ec2c7f4645a660a72720348316'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.783188 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e804b2ec2c7f4645a660a72720348316'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.798579 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e804b2ec2c7f4645a660a72720348316'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.816690 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-f8c17676-06ef-4098-83c9-590acb222436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e804b2ec2c7f4645a660a72720348316'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:04.830220 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1182/2369] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:14:04 2022] DELETE /identity/v3/users/e804b2ec2c7f4645a660a72720348316 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:14:05.190102 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:05.192251 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:05.212862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c3c88-54d0-42db-b0f8-4e84a8a8b92a admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:05.257395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:05.289889 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:05.290533 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.290813 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.291124 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] PATH_INFO: `/v3/projects/6038c843cf3d4268ba8953ca36cb5ad0` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.292257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6038c843cf3d4268ba8953ca36cb5ad0)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:05.294177 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:05.354044 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6038c843cf3d4268ba8953ca36cb5ad0', 'request_id': 'req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:05.364179 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6038c843cf3d4268ba8953ca36cb5ad0', 'request_id': 'req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:05.371187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6038c843cf3d4268ba8953ca36cb5ad0', 'request_id': 'req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:05.389597 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] The token cache is being invalidate because project 6038c843cf3d4268ba8953ca36cb5ad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:14:05.390380 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:05.393070 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-6bc457fb-d1fe-4a61-8abe-2e502396ae4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:05.396162 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1188/2370] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:14:05 2022] DELETE /identity/v3/projects/6038c843cf3d4268ba8953ca36cb5ad0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:14:05.404609 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5793903-a371-4786-8783-e9b82235ead5 None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.404956 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5793903-a371-4786-8783-e9b82235ead5 None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.405197 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5793903-a371-4786-8783-e9b82235ead5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.462152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-a5793903-a371-4786-8783-e9b82235ead5 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:05.463432 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-a5793903-a371-4786-8783-e9b82235ead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:05.529439 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1183/2371] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:14:05.538153 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce37dd5-74ce-4029-941e-249aaed69d9a None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.538508 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce37dd5-74ce-4029-941e-249aaed69d9a None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.538885 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce37dd5-74ce-4029-941e-249aaed69d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.566593 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-bce37dd5-74ce-4029-941e-249aaed69d9a None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:05.567861 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-bce37dd5-74ce-4029-941e-249aaed69d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:05.576446 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1189/2372] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:14:05.582351 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5793903-a371-4786-8783-e9b82235ead5 None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:05.592842 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5793903-a371-4786-8783-e9b82235ead5 None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:05.600602 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765a876-f38b-478a-93e3-0ed8c3a9dd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:05.601589 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765a876-f38b-478a-93e3-0ed8c3a9dd7b admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.602152 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765a876-f38b-478a-93e3-0ed8c3a9dd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.602378 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765a876-f38b-478a-93e3-0ed8c3a9dd7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.603082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765a876-f38b-478a-93e3-0ed8c3a9dd7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:14:05.603388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765a876-f38b-478a-93e3-0ed8c3a9dd7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:05.605250 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765a876-f38b-478a-93e3-0ed8c3a9dd7b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:05.618614 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1184/2373] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 19:14:05.623197 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce37dd5-74ce-4029-941e-249aaed69d9a None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:05.633113 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce37dd5-74ce-4029-941e-249aaed69d9a None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:05.638607 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aec2b-74f6-45f8-a590-ae1ee989e948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:05.638794 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aec2b-74f6-45f8-a590-ae1ee989e948 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.639270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aec2b-74f6-45f8-a590-ae1ee989e948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.639270 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aec2b-74f6-45f8-a590-ae1ee989e948 admin admin] PATH_INFO: `/v3/projects` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.639880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aec2b-74f6-45f8-a590-ae1ee989e948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:05.642396 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aec2b-74f6-45f8-a590-ae1ee989e948 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:05.686478 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1190/2374] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:14:05 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:14:05.688844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765a876-f38b-478a-93e3-0ed8c3a9dd7b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:05.702663 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765a876-f38b-478a-93e3-0ed8c3a9dd7b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:05.775882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8f455-b5ab-4920-85bd-ab1c7eb44e46 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.776312 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8f455-b5ab-4920-85bd-ab1c7eb44e46 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.776488 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8f455-b5ab-4920-85bd-ab1c7eb44e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.786958 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e8500-6a26-4cc4-b477-ebccbb74a318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:05.787697 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e8500-6a26-4cc4-b477-ebccbb74a318 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.788042 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e8500-6a26-4cc4-b477-ebccbb74a318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.788373 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e8500-6a26-4cc4-b477-ebccbb74a318 admin admin] PATH_INFO: `/v3/users` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.789062 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e8500-6a26-4cc4-b477-ebccbb74a318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:05.791554 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e8500-6a26-4cc4-b477-ebccbb74a318 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:05.795847 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-6af8f455-b5ab-4920-85bd-ab1c7eb44e46 None None] MFA Rules not processed for user `f7c55684a73c4fd49a629012f2d4d661`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:05.797140 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-6af8f455-b5ab-4920-85bd-ab1c7eb44e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:05.801362 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:05.817872 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1185/2375] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:14:05 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:14:05.821829 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e8500-6a26-4cc4-b477-ebccbb74a318 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:05.831511 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e8500-6a26-4cc4-b477-ebccbb74a318 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:05.836213 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf864d-22a8-431a-a1ea-1dd72742e46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:05.836805 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf864d-22a8-431a-a1ea-1dd72742e46b admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.837079 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf864d-22a8-431a-a1ea-1dd72742e46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.837352 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf864d-22a8-431a-a1ea-1dd72742e46b admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.837991 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf864d-22a8-431a-a1ea-1dd72742e46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:05.839604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf864d-22a8-431a-a1ea-1dd72742e46b admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:05.850474 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1186/2376] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:14:05.854807 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf864d-22a8-431a-a1ea-1dd72742e46b admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:05.856994 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1191/2377] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 19:14:05.865751 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf864d-22a8-431a-a1ea-1dd72742e46b admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:05.871401 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5e8f2-37ac-464e-aebf-8ef3ae843e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:05.872011 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5e8f2-37ac-464e-aebf-8ef3ae843e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.872285 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5e8f2-37ac-464e-aebf-8ef3ae843e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.872551 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5e8f2-37ac-464e-aebf-8ef3ae843e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.873210 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5e8f2-37ac-464e-aebf-8ef3ae843e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:05.874821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5e8f2-37ac-464e-aebf-8ef3ae843e78 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:05.886693 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1187/2378] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:14:05.891292 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8f455-b5ab-4920-85bd-ab1c7eb44e46 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:05.902347 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8f455-b5ab-4920-85bd-ab1c7eb44e46 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:05.907005 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784bb2b-0278-4f6c-813e-a078a0237006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:05.907686 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784bb2b-0278-4f6c-813e-a078a0237006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.908027 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784bb2b-0278-4f6c-813e-a078a0237006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.908257 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784bb2b-0278-4f6c-813e-a078a0237006 admin admin] PATH_INFO: `/v3/projects/1811b61359b74430b1e8dfd8257d9516/users/2fc2d063617f4d579a42aa4c36f46123/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.926246 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784bb2b-0278-4f6c-813e-a078a0237006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1811b61359b74430b1e8dfd8257d9516, user_id=2fc2d063617f4d579a42aa4c36f46123, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:05.928937 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784bb2b-0278-4f6c-813e-a078a0237006 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:05.943763 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1192/2379] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:14:05 2022] PUT /identity/v3/projects/1811b61359b74430b1e8dfd8257d9516/users/2fc2d063617f4d579a42aa4c36f46123/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:14:05.955241 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7e5e4-c0a7-4192-8896-1c127b49b16e None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:05.955620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7e5e4-c0a7-4192-8896-1c127b49b16e None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:05.955992 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7e5e4-c0a7-4192-8896-1c127b49b16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:05.991235 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-adf7e5e4-c0a7-4192-8896-1c127b49b16e None None] MFA Rules not processed for user `2fc2d063617f4d579a42aa4c36f46123`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:05.992497 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-adf7e5e4-c0a7-4192-8896-1c127b49b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:06.060571 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1188/2380] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:14:06.072463 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784bb2b-0278-4f6c-813e-a078a0237006 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:06.074277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-1784bb2b-0278-4f6c-813e-a078a0237006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:06.094501 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:06.143545 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784bb2b-0278-4f6c-813e-a078a0237006 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:06.147571 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3c27c-2b9e-4d32-a099-ff2c7ded89c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:06.148183 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3c27c-2b9e-4d32-a099-ff2c7ded89c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:06.148459 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3c27c-2b9e-4d32-a099-ff2c7ded89c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:06.148729 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3c27c-2b9e-4d32-a099-ff2c7ded89c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:06.158499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3c27c-2b9e-4d32-a099-ff2c7ded89c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:06.159969 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3c27c-2b9e-4d32-a099-ff2c7ded89c3 service cinder] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:06.169432 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1193/2381] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:14:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:14:06.192985 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7e5e4-c0a7-4192-8896-1c127b49b16e None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:06.194367 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-adf7e5e4-c0a7-4192-8896-1c127b49b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:06.252234 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7e5e4-c0a7-4192-8896-1c127b49b16e None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:06.255882 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe62e3f-06fb-4097-9668-6d07635caeed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:06.256478 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe62e3f-06fb-4097-9668-6d07635caeed service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:06.256752 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe62e3f-06fb-4097-9668-6d07635caeed service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:06.257274 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe62e3f-06fb-4097-9668-6d07635caeed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:06.259388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-cbe62e3f-06fb-4097-9668-6d07635caeed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:06.269525 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe62e3f-06fb-4097-9668-6d07635caeed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:06.271350 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe62e3f-06fb-4097-9668-6d07635caeed service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:06.280215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1189/2382] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:14:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:14:13.726222 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3c27c-2b9e-4d32-a099-ff2c7ded89c3 service cinder] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:13.814201 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3c27c-2b9e-4d32-a099-ff2c7ded89c3 service cinder] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:13.818668 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:13.819324 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:13.819603 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:13.819878 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] PATH_INFO: `/v3/users/2fc2d063617f4d579a42aa4c36f46123` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:13.821024 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc2d063617f4d579a42aa4c36f46123)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:13.822935 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:13.880499 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc2d063617f4d579a42aa4c36f46123'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:13.881133 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc2d063617f4d579a42aa4c36f46123'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:13.896332 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc2d063617f4d579a42aa4c36f46123'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:13.904066 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc2d063617f4d579a42aa4c36f46123'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:13.904790 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc2d063617f4d579a42aa4c36f46123'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:13.918001 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1194/2383] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:14:13 2022] DELETE /identity/v3/users/2fc2d063617f4d579a42aa4c36f46123 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:14:13.923397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66182f0-2d50-47d3-8f99-12266413164d None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:13.923999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66182f0-2d50-47d3-8f99-12266413164d None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:13.923999 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66182f0-2d50-47d3-8f99-12266413164d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:13.948442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-d66182f0-2d50-47d3-8f99-12266413164d None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:13.949871 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d66182f0-2d50-47d3-8f99-12266413164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:14.014728 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1190/2384] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:14:14.025200 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:14.026971 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:14.074776 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:14.096608 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055125-9553-4418-8e8f-3e863e8557d6 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:14.100576 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d97eb-dfa6-463c-b4b6-44b3055c97ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:14.101182 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d97eb-dfa6-463c-b4b6-44b3055c97ba service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.101456 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d97eb-dfa6-463c-b4b6-44b3055c97ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.101726 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d97eb-dfa6-463c-b4b6-44b3055c97ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:14.111403 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d97eb-dfa6-463c-b4b6-44b3055c97ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:14.112873 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d97eb-dfa6-463c-b4b6-44b3055c97ba service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:14.122340 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1195/2385] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:14:14.415445 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66182f0-2d50-47d3-8f99-12266413164d None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:14.417052 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-d66182f0-2d50-47d3-8f99-12266413164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:14.424140 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:14.430155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66182f0-2d50-47d3-8f99-12266413164d None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:14.435120 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:14.435750 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.436030 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.436306 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] PATH_INFO: `/v3/projects/1811b61359b74430b1e8dfd8257d9516` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:14.437417 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1811b61359b74430b1e8dfd8257d9516)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:14.441098 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:14.488198 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1811b61359b74430b1e8dfd8257d9516', 'request_id': 'req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:14.494510 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1811b61359b74430b1e8dfd8257d9516', 'request_id': 'req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:14.503215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1811b61359b74430b1e8dfd8257d9516', 'request_id': 'req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:14.513766 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] The token cache is being invalidate because project 1811b61359b74430b1e8dfd8257d9516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:14:14.514049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:14.514659 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-ea295dfc-26b1-4ba5-a1d5-c5d2799e0c3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:14.516825 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1191/2386] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:14:14 2022] DELETE /identity/v3/projects/1811b61359b74430b1e8dfd8257d9516 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:14:14.521728 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea8ad-fe4c-4dea-a2cb-98f26bc10cc9 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.522015 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea8ad-fe4c-4dea-a2cb-98f26bc10cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.522345 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea8ad-fe4c-4dea-a2cb-98f26bc10cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:14.544090 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-84aea8ad-fe4c-4dea-a2cb-98f26bc10cc9 None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:14.545286 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-84aea8ad-fe4c-4dea-a2cb-98f26bc10cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:14.605034 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1196/2387] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:14:14.672465 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92303a71-45fe-4372-b2d8-3bf6c8c4653b None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.672735 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92303a71-45fe-4372-b2d8-3bf6c8c4653b None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.673057 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92303a71-45fe-4372-b2d8-3bf6c8c4653b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:14.694001 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-92303a71-45fe-4372-b2d8-3bf6c8c4653b None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:14.696280 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-92303a71-45fe-4372-b2d8-3bf6c8c4653b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:14.704003 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1192/2388] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:14:14.708395 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea8ad-fe4c-4dea-a2cb-98f26bc10cc9 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:14.718574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea8ad-fe4c-4dea-a2cb-98f26bc10cc9 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:14.723277 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9447-c725-4091-9c17-1a3fd971c230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:14.723882 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9447-c725-4091-9c17-1a3fd971c230 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.724163 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9447-c725-4091-9c17-1a3fd971c230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.724438 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9447-c725-4091-9c17-1a3fd971c230 admin admin] PATH_INFO: `/v3/domains` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:14.725085 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca9447-c725-4091-9c17-1a3fd971c230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 19:14:14.725390 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca9447-c725-4091-9c17-1a3fd971c230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:14.727170 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca9447-c725-4091-9c17-1a3fd971c230 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:14.743057 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1197/2389] 10.222.0.58 () {58 vars in 1153 bytes} [Fri Oct 28 19:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 19:14:14.747548 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92303a71-45fe-4372-b2d8-3bf6c8c4653b None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:14.758009 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92303a71-45fe-4372-b2d8-3bf6c8c4653b None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:14.763203 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe5261-4b58-4c40-a991-b47c247474d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:14.763833 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe5261-4b58-4c40-a991-b47c247474d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.764110 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe5261-4b58-4c40-a991-b47c247474d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.764382 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe5261-4b58-4c40-a991-b47c247474d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:14.765049 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe5261-4b58-4c40-a991-b47c247474d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:14.767705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe5261-4b58-4c40-a991-b47c247474d1 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:14.804861 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1193/2390] 10.222.0.58 () {60 vars in 1153 bytes} [Fri Oct 28 19:14:14 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:14:14.806326 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9447-c725-4091-9c17-1a3fd971c230 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:14.817203 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9447-c725-4091-9c17-1a3fd971c230 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:14.876107 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33580e1d-2a03-4f1b-bfb9-efbef44b02a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:14.876630 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33580e1d-2a03-4f1b-bfb9-efbef44b02a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.876934 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33580e1d-2a03-4f1b-bfb9-efbef44b02a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.877204 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33580e1d-2a03-4f1b-bfb9-efbef44b02a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:14.877828 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33580e1d-2a03-4f1b-bfb9-efbef44b02a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:14.879817 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33580e1d-2a03-4f1b-bfb9-efbef44b02a7 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:14.900880 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1198/2391] 10.222.0.58 () {60 vars in 1144 bytes} [Fri Oct 28 19:14:14 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 19:14:14.904931 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe5261-4b58-4c40-a991-b47c247474d1 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:14.915726 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe5261-4b58-4c40-a991-b47c247474d1 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:14.920303 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c03cd6-db2f-4192-9cd2-f51aa8e5abda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:14.920894 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c03cd6-db2f-4192-9cd2-f51aa8e5abda admin admin] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.921171 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c03cd6-db2f-4192-9cd2-f51aa8e5abda admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.921442 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c03cd6-db2f-4192-9cd2-f51aa8e5abda admin admin] PATH_INFO: `/v3/roles` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:14.922588 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c03cd6-db2f-4192-9cd2-f51aa8e5abda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:14.924215 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c03cd6-db2f-4192-9cd2-f51aa8e5abda admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:14.934232 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1194/2392] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 19:14:14.938837 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33580e1d-2a03-4f1b-bfb9-efbef44b02a7 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:14.950279 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33580e1d-2a03-4f1b-bfb9-efbef44b02a7 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:14.955096 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bf0f0-3c86-4c80-ad82-b2cbe5c01f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:14.955688 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bf0f0-3c86-4c80-ad82-b2cbe5c01f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.955962 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bf0f0-3c86-4c80-ad82-b2cbe5c01f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.956231 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bf0f0-3c86-4c80-ad82-b2cbe5c01f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:14.956862 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bf0f0-3c86-4c80-ad82-b2cbe5c01f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:14.958480 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bf0f0-3c86-4c80-ad82-b2cbe5c01f28 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:14.969253 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1199/2393] 10.222.0.58 () {58 vars in 1122 bytes} [Fri Oct 28 19:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 19:14:14.973594 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c03cd6-db2f-4192-9cd2-f51aa8e5abda admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:14.983898 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c03cd6-db2f-4192-9cd2-f51aa8e5abda admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:14.989358 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5e340-8b2c-4cf7-9d32-a24118e58fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:14.990000 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5e340-8b2c-4cf7-9d32-a24118e58fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:14.990281 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5e340-8b2c-4cf7-9d32-a24118e58fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:14.990572 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5e340-8b2c-4cf7-9d32-a24118e58fbb admin admin] PATH_INFO: `/v3/projects/2690188980a248cba5744808460081e4/users/85d0e91f0370415e82760d6dae2c9168/roles/3c56cfa91fba453da353f21d7d6c1189` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:15.007321 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5e340-8b2c-4cf7-9d32-a24118e58fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2690188980a248cba5744808460081e4, user_id=85d0e91f0370415e82760d6dae2c9168, role_id=3c56cfa91fba453da353f21d7d6c1189)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:15.010109 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5e340-8b2c-4cf7-9d32-a24118e58fbb admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:15.023451 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1195/2394] 10.222.0.58 () {60 vars in 1483 bytes} [Fri Oct 28 19:14:14 2022] PUT /identity/v3/projects/2690188980a248cba5744808460081e4/users/85d0e91f0370415e82760d6dae2c9168/roles/3c56cfa91fba453da353f21d7d6c1189 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 19:14:15.033796 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5cabd-47df-4b23-93f5-5734d843a272 None None] REQUEST_METHOD: `POST` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:15.034101 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5cabd-47df-4b23-93f5-5734d843a272 None None] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:15.034373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5cabd-47df-4b23-93f5-5734d843a272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:15.064138 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.auth.core [None req-97a5cabd-47df-4b23-93f5-5734d843a272 None None] MFA Rules not processed for user `85d0e91f0370415e82760d6dae2c9168`. Rule list: `[]` (Enabled: `True`). {{(pid=81363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:15.065373 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-97a5cabd-47df-4b23-93f5-5734d843a272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:15.099598 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:15.133753 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1200/2395] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:14:15.145101 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5e340-8b2c-4cf7-9d32-a24118e58fbb admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:15.146762 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-daa5e340-8b2c-4cf7-9d32-a24118e58fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:15.203082 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5e340-8b2c-4cf7-9d32-a24118e58fbb admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:15.206768 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0120d0-a458-4258-8c9c-6ba9bdca124e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302ded4fdd72405c8ce8a364f7d2400c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:15.207397 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0120d0-a458-4258-8c9c-6ba9bdca124e service cinder] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:15.207705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0120d0-a458-4258-8c9c-6ba9bdca124e service cinder] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:15.207975 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0120d0-a458-4258-8c9c-6ba9bdca124e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:15.216205 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0120d0-a458-4258-8c9c-6ba9bdca124e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:15.217620 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0120d0-a458-4258-8c9c-6ba9bdca124e service cinder] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:15.226729 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1196/2396] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:14:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:14:25.130484 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=11 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:25.222844 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:25.426346 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5cabd-47df-4b23-93f5-5734d843a272 None None] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:25.490887 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5cabd-47df-4b23-93f5-5734d843a272 None None] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:25.495510 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:25.496144 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:25.496423 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:25.496694 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] PATH_INFO: `/v3/users/85d0e91f0370415e82760d6dae2c9168` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:25.497820 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d0e91f0370415e82760d6dae2c9168)` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:25.499805 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:25.551725 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d0e91f0370415e82760d6dae2c9168'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:25.552387 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d0e91f0370415e82760d6dae2c9168'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:25.564660 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d0e91f0370415e82760d6dae2c9168'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:25.572574 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d0e91f0370415e82760d6dae2c9168'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:25.573309 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.notifications [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d0e91f0370415e82760d6dae2c9168'} {{(pid=81363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:25.587737 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1201/2397] 10.222.0.58 () {58 vars in 1224 bytes} [Fri Oct 28 19:14:25 2022] DELETE /identity/v3/users/85d0e91f0370415e82760d6dae2c9168 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:14:25.592604 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff73ec-f2a8-42ae-a626-2fb95f72abda None None] REQUEST_METHOD: `POST` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:25.592879 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff73ec-f2a8-42ae-a626-2fb95f72abda None None] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:25.593145 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff73ec-f2a8-42ae-a626-2fb95f72abda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:25.614407 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.auth.core [None req-33ff73ec-f2a8-42ae-a626-2fb95f72abda None None] MFA Rules not processed for user `d171ff8c81304d799c869856dff7ee6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 19:14:25.615682 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-33ff73ec-f2a8-42ae-a626-2fb95f72abda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:25.679100 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1197/2398] 10.222.0.58 () {58 vars in 958 bytes} [Fri Oct 28 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 19:14:25.689765 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:25.691258 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:25.781263 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be00b4-d7ca-44fe-aee0-1dd4b49efcd2 admin admin] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:25.785566 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a00703-9046-45ab-afb5-3f51dcdf7a04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:25.786187 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a00703-9046-45ab-afb5-3f51dcdf7a04 service neutron] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:25.786460 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a00703-9046-45ab-afb5-3f51dcdf7a04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:25.786727 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a00703-9046-45ab-afb5-3f51dcdf7a04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:25.797404 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a00703-9046-45ab-afb5-3f51dcdf7a04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:25.799290 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a00703-9046-45ab-afb5-3f51dcdf7a04 service neutron] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:25.810243 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1202/2399] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:14:26.104924 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff73ec-f2a8-42ae-a626-2fb95f72abda None None] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:14:26.106346 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-33ff73ec-f2a8-42ae-a626-2fb95f72abda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:14:26.119563 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff73ec-f2a8-42ae-a626-2fb95f72abda None None] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:14:26.125300 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd171ff8c81304d799c869856dff7ee6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b28e4680a856417f99f9b94c78ac743d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:14:26.126058 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:14:26.126408 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:14:26.126757 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] PATH_INFO: `/v3/projects/2690188980a248cba5744808460081e4` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:14:26.128288 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2690188980a248cba5744808460081e4)` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:14:26.130388 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:14:26.182257 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2690188980a248cba5744808460081e4', 'request_id': 'req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:26.189010 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2690188980a248cba5744808460081e4', 'request_id': 'req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:26.197428 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2690188980a248cba5744808460081e4', 'request_id': 'req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99'} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:26.210172 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] The token cache is being invalidate because project 2690188980a248cba5744808460081e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 19:14:26.210518 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:26.211292 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.notifications [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 19:14:26.212892 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1198/2400] 10.222.0.58 () {58 vars in 1233 bytes} [Fri Oct 28 19:14:26 2022] DELETE /identity/v3/projects/2690188980a248cba5744808460081e4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 19:14:35.805236 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:14:36.206705 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:16:11.710142 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a00703-9046-45ab-afb5-3f51dcdf7a04 service neutron] Authenticating user token {{(pid=81363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:16:11.711895 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-c7a00703-9046-45ab-afb5-3f51dcdf7a04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:16:11.818859 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a00703-9046-45ab-afb5-3f51dcdf7a04 service neutron] Validating token access rules against request {{(pid=81363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:16:11.878781 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c09d5-87c6-4b4a-96b9-91eb697e5013 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da958657ac3041848d24a76b76705489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:16:11.879421 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c09d5-87c6-4b4a-96b9-91eb697e5013 service placement] REQUEST_METHOD: `GET` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:16:11.879696 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c09d5-87c6-4b4a-96b9-91eb697e5013 service placement] SCRIPT_NAME: `/identity` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:16:11.879965 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c09d5-87c6-4b4a-96b9-91eb697e5013 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:16:11.881987 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.fernet_utils [None req-529c09d5-87c6-4b4a-96b9-91eb697e5013 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:16:11.893532 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c09d5-87c6-4b4a-96b9-91eb697e5013 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:16:11.895172 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c09d5-87c6-4b4a-96b9-91eb697e5013 service placement] RBAC: Authorization granted {{(pid=81363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:16:11.905658 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: [pid: 81363|app: 0|req: 1203/2401] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:16:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:16:21.900890 dv-cinder-862365-1-iscsi devstack@keystone.service[81363]: DEBUG dbcounter [-] [81363] Writing DB stats keystone:SELECT=67 {{(pid=81363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 19:17:14.178954 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] Authenticating user token {{(pid=81364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 19:17:14.180422 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:17:14.240581 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b17d-97be-4e0b-8d4c-5d5d600ebc99 admin admin] Validating token access rules against request {{(pid=81364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 19:17:14.244155 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f7e60-f07e-4a0d-a13b-cb58e05a1593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffb662f4104e349c81c578bf66ab7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a389b8e27a4c468491201f54ef9614', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 19:17:14.244754 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f7e60-f07e-4a0d-a13b-cb58e05a1593 service neutron] REQUEST_METHOD: `GET` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 19:17:14.245033 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f7e60-f07e-4a0d-a13b-cb58e05a1593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 19:17:14.245329 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f7e60-f07e-4a0d-a13b-cb58e05a1593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 19:17:14.247102 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.fernet_utils [None req-6e7f7e60-f07e-4a0d-a13b-cb58e05a1593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 19:17:14.256821 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f7e60-f07e-4a0d-a13b-cb58e05a1593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 19:17:14.258332 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f7e60-f07e-4a0d-a13b-cb58e05a1593 service neutron] RBAC: Authorization granted {{(pid=81364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 19:17:14.267036 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: [pid: 81364|app: 0|req: 1199/2402] 10.222.0.58 () {60 vars in 1365 bytes} [Fri Oct 28 19:17:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 19:17:24.263371 dv-cinder-862365-1-iscsi devstack@keystone.service[81364]: DEBUG dbcounter [-] [81364] Writing DB stats keystone:SELECT=26 {{(pid=81364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}